 |
 |
|
Technology Overview Table of Contents The Starta Enterprise System (SES) Fundamentally, the Starta Enterprise System (SES) is a web-based system for managing data and documents. We've leveraged these base capabilities to provide many more advanced features in the areas of Document Management, Customer Relationship Management, and Report Automation, to name a few. Our original target market was the Affordable Housing Industry, a small niche full of many complex business practices and government requirements. In developing a solution robust enough to meet the needs of this industry, we invented the Business Operating System, which we have since applied to other industries and purposes. Though we have branched out over the years, the Affordable Housing Industry remains one of our primary foci. (Back to TOC) The Business Operating System (BOS) The underlying framework of the Starta Enterprise System is an extensible web platform designed and built by Starta, which we call the Business Operating System (BOS). The BOS includes:
Code Base Modularity As of early 2006, SES consisted of approximately 500,000 lines of code for approximately 2,800 pages, and many different back-end support systems. All of our code is written in C#, JavaScript, or T-SQL, with the majority in C#. You might notice that these figures indicate an average of less than 180 lines of code per page; this is due to our extremely modularity. We actually have around 900 front-end classes (in C#), each between 500-1000 lines on average, which generate between 1-9 "pages" each. This modularity benefits our customers by letting us provide new functionality faster, and with amazingly few bugs compared to most systems of our size. A system comparable to ours, but without our modularity, could easily be several million lines of code, which means many more hours in coding and testing. Our modularity also lets use use a development team that is 2-3 times smaller than it would otherwise have to be, therefore allowing us to keep our prices lower, among other things. (Back to TOC) MSSQL 2008 Database Platform Our back-end data is stored in two Microsoft SQL Server 2008 databases: one for the bulk of our data, and another for certain custom procedures and dynamic tables. Client data is secured through our proprietary business logic, which insures that users only see what they have been granted access to. SQL is not an Object-Oriented (OOP) programming language, but we have such a strong OOP mentality that we have done much to mimic that style of functionality within T-SQL Stored Procedures. The result is that we only have about 1500 procedures and fewer than 100 UDFs as of early 2006, and we do not have to change a single procedure in order to implement one of our "plug-in" modules in a new location within our system. (Back to TOC) IHttpHandler and the .NET 2.0 Framework The .NET Framework is not required on client machines to run any of our web-based content;it is what powers the business logic and user interface layers on our front-end servers. Shortly after the .NET Framework 2.0 went gold in November 2005, we adopted it as our standard. The earliest versions of our system were in ASP 2.0, but in June 2003 we completed the switch to ASP.NET 1.0. You may have noticed the .sh extension used by our system; this stands for "Starta Handler." This is a custom extension which is serviced by custom IHttpHandlers. All of our controls are stored in DLLs hosted in IIS, but do not use the bulk of the ASP.NET runtime, or any ASP.NET controls. We've coded each and every one of our controls from scratch, from Tree Views to Sortable/Filterable/Paginatable Listings to mutable Flow Charts. We have therefore been able to streamline everything, so that a minimum amount of processing is done on both the client machine and the server, and the least possible amount of content has to be transmitted across the wire from our servers to your computer. If you've noticed how fast our system responds even on cheap DSL lines, this is why. (Back to TOC) DHTML and Very Small Page Size There is currently a trend towards AJAX and XML, which we respect, but we have perfected a different approach. While we certainly do use XML throughout our system for various purposes (such as for the Weather.com feed), we do not use it for anything client-side. All of our server-side controls emit finely-tuned DHTML, which relies on external CSS and JavaScript files so that we embed as little code as possible in the actual page itself. There are a number of benefits to this approach, but the largest is that it is the fastest way by far. Using our approach, the average page listing of 100 items is less than 20kb, and the average document cabinet is less than 50kb (and we have a "fast cabinet" to make this download even smaller). Note that these are some of the largest pages within our system; most are vastly smaller than that. Even better, once this data is downloaded by the browser, there is usually no post-processing needed. (Back to TOC) Supported Browsers For many years, we supported only Internet Explorer, but now this has all changed. Mozilla FireFox 1.5 or greater is now fully supported by the system, and is now our recommended browser platform due to FireFox's increased speed and reliability. The recommended version of Internet Explorer is version 6.0, although the system mostly works in 5.5 as well. There are some minor graphical anomolies that occur using version 5.5 of this browser. IE on the Macintosh does not work at all with our system, so Mac users will need to use the latest version of Firefox. Netscape only works with our system when run on Windows using the "Display Like Internet Explorer" option. Using this option, of course, basically means that Internet Explorer is used in the first place. The "Display Like Firefox" option in Netscape is not presently supported, as it renders like FireFox 1.0 rather than FireFox 1.5. When Netscape updates to the latest version of the Gecko engine, they will be fully supported by our system. Why don't we support more browsers? Unlike simple web pages, our system has a great deal of interconnected functionality which pushes the functionality of browsers to the max. We've spent a great deal of time making sure that both the Gecko and Internet Explorer engines are fully supported, as these are what the vast majority of Internet users are running. We are considering extending support to other browsers (such as Opera) as well, but there is a long list of features which have to be supported by a browser for a system like ours to function properly. (Back to TOC) Back-end Support Systems There are a variety of completely custom support services which are part of the BOS. These subsystems are never directly accessed by end users, but work behind the scenes to make the user experience what it is. All of these subsystems are presently in a single IIS-hosted DLL, which runs separate threads for most of the systems. The System Agent is the oldest subsystem within SES, though it has been updated and upgraded many times since it was first created. The System Agent is our main schedule execution tool. It polls every five minutes, and runs jobs at scheduled intervals. Jobs that occur on a five-minute interval include work like expiring idle sessions, and jobs on a two-hour interval include re-caching dirty account security caches. The most common jobs are those which run at 8:30 PM, and 2:00 AM. All of the "nightly/daily" emails are sent out at 8:30 PM, and all of the routine cleanup and full cache refreshing operations occur at 2:00 AM. The System Agent only runs on one of our servers at a time. The Action Queue is nearly as old as the System Agent, and is one of our primary methods for undertaking asynchronous operations. The Action Queue is similar to other queuing solutions like MSMQ, but it is much lighter weight and includes some functionality that was not available anywhere else. Each front-end server has a separate running Action Queue, and tab-delimited commands are dumped into the queue at various times due to activity in the front-end. The Action Queue polls every ten seconds, and executes any items it finds in the queue using .NET Reflection. The Late Writer Queue is very new to the system, and is an extension of the Action Queue. The Late Writer executes on a much slower interval than the Action Queue--every 8 hours, in fact. We use this on our corporate website to buffer the database against floods of hits which may be generated by website visitors. The Asynchronous Action Queue is a SQL-based alternative to our traditional file-based Action Queue. Sometimes SQL Stored Procedures need to asynchronously invoke C#-based methods (for logging or emailing purposes, most times), and allows them to do that. The exact same tab-delimited format is used from the Action Queue, but it is stored in a SQL table accessible to our database procedures. Normally, Asynchronous Action Queue items are picked up by the System Agent every five minutes, but the system front-end has the option to force processing of the queue at any time. This is generally done after the front-end of the system executes database procedures which it suspects might have made entries in the Asynchronous Action Queue, and which it wants to execute faster than up-to five minutes later. Like the normal Action Queue, these methods are invoked using Reflection in .NET. The Starta Service Provider has been around for a few years, and is an internal web service provider. Specifically, the Service Provider provides a binary TCP .NET Remoting interface, which is used for two purposes. First, this is a way to execute asynchronous commands immediately, for cases when the up-to 10 second delay of the Action Queue is just too slow. Moving Process Tracking Occurrences to the next stage is an excellent example of this. The second use for the Service Provider is to run processor-intensive tasks on a backup server, to avoid bogging down any of the servers in the primary queue. This may be done synchronously or asynchronously. The best example of when we do synchronous processing using this component is for the "post processing" of reports, which can be both processor and memory intensive. The Mail Queue is a recently-added service which helps to insure that mail messages are always sent. Mail messages are sent at various times by the System Agent, Action Queue, Starta Service Provider, and Asynchronous Action Queue, and all of their mail messages are routed through Starta's MS Exchange 2003 server, which is located at our corporate offices rather than our production site. There is almost never a problem with this setup, but occasionally we need to reboot our Exchange server for upgrades or other maintenance, and sometimes issues arise with our corporate network's connectivity (our production site is vastly more fault-tolerant, by design.) When these kinds of issues arise, the production environment is wholly unaffected, with one exception--it is unable to send mail because its outbound SMTP server is suddenly unavailable. In these rare instances, the mail messages are dumped into our own proprietary Mail Queue on the originating server. This service polls every minute, and tries to send any messages it discovers in its queue. If the SMTP server is still unavailable, it waits another minute and then tries again, until the SMTP server again becomes available. SMTP server unavailability is very rare for us, but this service insures that we can absorb a brief amount of mail server downtime without causing any mail messages to be permanently lost. The Query Cache is another recently-added service, and it allows for the results of database queries to be serialized to disk for later reuse. This is particularly useful for our corporate website, which uses the database to find out things such as the latest version of the system, the version history, etc. These values rarely change, so to avoid unnecessary hits against the database, the Query Cache is employed so that the front-end servers servicing the Starta website can simply read from an on-disk cache. (Back to TOC) Export Capabilities We have the capability to export data in a variety of formats, and we have the ability to quickly add support for other formats if the need should arise. Starta currently exports data in tab-delimited, Excel 2003 XML format, BIFF8 (Excel 97 and up), Word 97 and up, iCalendar appointments, Microsoft vCard, and serialized .NET objects. We can also export into custom XML schemas, and can provide Remoting or SOAP web service access if need be (although we tend to use raw text transfer over HTTP for our own purposes--as in Starta Optimized). Usually these exports are in the form of reports, but we have the capability to develop other solutions for your company if need be. (Back to TOC) Import Capabilities If you have an existing system, or a lot of data in some other data format, it can be imported into our system. We are experts at importing from MySQL/MSSQL and MS Access databases, and from Excel files and tab-delimited files. Other formats, such as DB2, are also possible, although slightly less preferred. Many other file formats can be exported or converted to the formats already mentioned; for example, ACT databases can be converted to Excel and then easily consumed. As a general note, data import often requires a substantial number of skilled hours to do, especially if the data isn't "clean" (consistent, correct). We can always do imports, but if there is a low row count or an especially high column count, then it is often more cost-effective to manually enter the data. We offer data entry services for a fee, for when these needs arise. (Back to TOC) Custom Technologies We have put a lot of work into developing custom solutions that solve the problems of enterprise-class applications more efficiently than the components sold in the mass market. Many people believe that you shouldn't "reinvent the wheel," and we agree, but we've almost always found favorable performance, price, and reliability gains by developing our own parts. If you've been using "custom" systems built entirely out of pre-fab purchased components, you'll immediately see a positive difference with us. Among the more notable of our solutions are the following, although these are only a sampling: Our Session Engine is lightweight, holds a substantial amount of data, and is fully cluster-ready. There is a lot of contextual information for each logged-in user, including cached security rights for their current location(s) in the system, and all of this is handled with lightning-fast precision that is not possible with a more generalized session solution. The SES Security Model is one of the most important features of the BOS. Storage, memory, and processing requirements are as minimal as possible, and the model is as flexible as you could imagine. Key features include overrideable defaults, robust dimensioning (as in a data cube), task-based rights assigned to roles, and person-based or team-based assignments to roles. We actually have three kinds of tasks within the system: defined (following the model above), derived (based on business logic assessments), and document (based on an extension of the model above). Our Compact DataSets (CDataSets) are much less intensive to load than normal .NET Datasets, and do not have to be disposed. The lack of disposal requirement alone removes a potential source of memory leaks, and the smaller size of our CDataSets makes the system speedier. We have full serialization support (as noted by the Query Cache), and we use our standard serialization versioning methodology so that we never have versioning conflicts (which exist between .NET DataSets in 1.0 and 1.1). Of course, our CDataSets do not have a lot of the functionality of standard DataSets, such as databinding, sorting, or a host of other things--but these are not features we need for our purposes. We've built a custom Query Builder tool which allows us to generate vertically and horizontally secured database queries on the fly. This tool is very specific to our own system and its security model, but within this specific environment it is very fast and quite functional. This is what makes our Ad-Hoc report engine possible. We also have a custom Table Manager tool which allows us to create and alter SQL tables on the fly. This is what maintains the custom tables behind such modules as Public Applications, Period Data Forms, and the dynamic fields on Workspaces. This tool means we do not have to either manually manage the tables behind the scenes (not feasible in the long run), or use a system of pivot tables with generic data types (which is slower and uses 2-3 times more disk space). We have developed many different Custom User Interface Elements, the bulk of which work in both IE and Firefox. Our checkboxes and radio buttons are completely custom not just in appearance, but in back-end function, too. We are actually able to transmit much less data for a form post with 300 checkboxes in it than the standard W3C standard normally allows for. Another notable achievement is our virtual window system, which operates much faster than traditional window.open, popup, or modal/modeless dialogs--and which is not not affected by popup blockers. Don't worry, though: we never show advertisements. There are many other custom elements which make the user experience easier, such as our very efficient tree views, our listings with headers that slide to stay in view, panel buttons which slide to stay in view, and a draggable flow chart engine. We built our own TWAIN Scanning Software in a product we called Starta Document. Starta Document was decommissioned in mid-2005, but the TWAIN algorithms live on. We'll be building TWAIN Scanning support into Starta Optimized in 2006 or 2007. We have made a Database Code Export tool that allows us to safely and quickly export our database Stored Procedures, UDFs, and Triggers. Not only does it do the export in only 2 seconds (as opposed to the twenty seconds that SQL Enterprise Manager takes), the SQL generated by the export can be applied in less than a minute (compared to the nearly two minutes taken by the standard approach). Triggers also cannot normally be scripted by Enterprise Manager except as part of the table they are on, and extra whitespace lines are added at the top and bottom of all code exported by Enterprise Manager, which gets overwhelming over time as it adds up. One last key improvement of our code export tool is that it only drops each object directly before it creates it, so that only one object is missing at a time. If few users are on the system (such as during lunch), small amounts of code can be imported into production without any negative impact on users. We have made an Automatic Lookup Updates Generation tool that allows us to auto-script values which have been added to lookup tables in development, so that these values can be added to production. There are many DBAs that frown on the term "lookup table," but in our system we use the term for one very specific meaning: tables which contain data that is only ever updated by our programmers--never by a user or any Stored Procedure. This tool allows us to make updates via any tool in development, and have our entries nicely scripted for import into production when we next do a release. (Back to TOC) Third-Party Components While we generally prefer to build everything ourselves, we know to stick to our strengths. We use several third party components for parts of the system which would simply have been too time-consuming or costly to develop ourselves, or which were outside our area of expertise. We use the InnovaStudio WYSIWYG Editor for our HTML-editing needs. We had developed an IE-only HTML editor of our own in 2003, but the InnovaStudio solution is much faster than what we came up with, has a lot more features, and works in both IE and Firefox. Excel exporting is one of the key parts of our reporting capabilities, and for this we use Aspose.Cells, a powerful tool for excel manipulation. Through this tool we are able to create Excel graphs and formulas, and do all sorts of non-linear display techniques. This tool is also extremely fast at doing more linear template-based exports to Excel, which is the manner in which we structure most of our reports. Word exporting is the other basic part of our reporting capabilities, and for this we use Aspose.Words, which allows for many advanced manipulations of existing word documents, creation of new documents, import of HTML into word format, and more. For PDF export and manipulation, we use PDFTron PDFNet, a very fast component which also has support for PDF Rasterizing (for viewing PDFs), something which many PDF component vendors do not offer. (Back to TOC) Diagrams and Other Documents
(Back to TOC) |
|
