Informant Spotlight

MIDAS 3 / Distributed Applications / Multi-tier Applications

By Dan Miser

MIDAS 3

Delivers Robust, Efficient, Multi-tier Applications

MIDAS has grown and matured considerably since its initial release two years ago. The excitement of being able to distribute data and create thin clients was overpowering. With each release, new features have been added to make it easier to create more complex applications and solve business problems more efficiently. MIDAS 3 proves to be no exception.

Note: This article was written with a pre-release copy of Delphi 5. Features are subject to change prior to the official release of Delphi 5. Consult the manual for confirmation of the features that made it into the release version. The code for this article will be made available shortly after the release of Delphi 5 at the Informant Web site (see end of article for details) and at http://www.execpc.com/~dmiser.

New Events, Properties, and Components

MIDAS 3 introduces several new components, properties, and events. See the table in Figure 1 for a brief overview of these new features. However, there are more changes to MIDAS than first appear when unwrapping the package. The big changes we’ll cover in this article are:

§ No more IProvider

§ No more TProvider

§ TSocketConnection changes

§ RDM pooling

§ Threaded server changes

§ Introducing TWebConnection

§ Web MIDAS client

§ Miscellaneous changes

Name	Description
TWebConnection	Allows MIDAS traffic to be sent using HTTP.
TXMLBroker	Interfaces between a MIDAS application server and MidasPageProducer.
TMidasPageProducer	Provides the ability to use MIDAS application servers from HTML.
TSocketConnection. SupportCallbacks	Specifies whether or not callbacks are supported for TSocketConnection.
TCustomProvider
Exported	Identifies which providers are exported from the RDM.
OnGetTableName	Ease-of-use event to specify which table will be updated.
TCustomProvider	Allows persistent state information to be ...
AfterApplyUpdates	sent to the client.
AfterExecute	sent to the client.
AfterGetParams	sent to the client.
AfterGetRecords	sent to the client.
AfterRowRequest	sent to the client.
BeforeApplyUpdates	retrieved from the client.
BeforeExecute	retrieved from the client.
BeforeGetParams	retrieved from the client.
BeforeGetRecords	retrieved from the client.
BeforeRowRequest	retrieved from the client.
TCustomProvider.Options
poAllowMultiRecordUpdates	Permits multiple records to be updated at once.
poDisableInserts	Stops inserts from happening on the client.
poDisableEdits	Stops edits from happening on the client.
poDisableDeletes	Stops deletes from happening on the client.
poNoReset	Ignores flag for resetting provider.
poAutoRefresh	Automatically refreshes the ClientDataset after ApplyUpdates.
poPropogateChanges	Sends changes that occur on the application server back to the client and merges them.
poAllowCommandText	Ability to send new SQL from the client to the server.
TClientDataset	Allows persistent state information to be ...
AfterApplyUpdates	retrieved from the server.
AfterExecute	retrieved from the server.
AfterGetParams	retrieved from the server.
AfterGetRecords	retrieved from the server.
AfterRowRequest	retrieved from the server.
BeforeApplyUpdates	passed to the server.
BeforeExecute	passed to the server.
BeforeGetParams	passed to the server.
BeforeGetRecords	passed to the server.
BeforeRowRequest	passed to the server.

Figure 1: New components, properties, and events found in MIDAS 3.

No More IProvider

While new components are always great, the most significant change found in MIDAS 3 is the removal of the IProvider interface. IProvider has been central to all MIDAS applications since MIDAS first hit the streets. You would export the IProvider interface of a TDBDataset from a Remote DataModule (RDM), and the client and server applications would talk to each other using the IProvider interface.

While this was a good model, it introduced the overhead of “state” in your application. A stateful application is one that remembers something about the previous call. For example, if you set ClientDataset.PacketRecords=5, the server needed to keep track of how many records have been sent to each client, introducing a stateful application. In contrast, stateless programming is on everyone’s mind these days. With Microsoft preaching stateless programming as a way of life for MTS components, it’s bound to get even more exposure.

So, with IProvider out of the picture, how do client and server communicate? Simple. We’ll use the IAppServer interface instead. Looking at the listing in Figure 2, you’ll notice many similarities between IProvider and IAppServer. The big difference is that IAppServer reduces the number of round-trip calls required to complete any given task, and it has no state information. You can still maintain state information on the client, but the server will be stateless. This is key for environments such as MTS that all but require stateless programming.

type

  IAppServer = interface(IDispatch)

    ['{ 1AEFCC20-7A24-11D2-98B0-C69BEB4B5B6D }']

function AS_ApplyUpdates(const ProviderName: WideString;

    Delta: OleVariant; MaxErrors: Integer;

out ErrorCount: Integer; var OwnerData: OleVariant):

    OleVariant; safecall;

function AS_GetRecords(const ProviderName: WideString;

    Count: Integer; out RecsOut: Integer; Options: Integer;

const CommandText: WideString; var Params: OleVariant;

var OwnerData: OleVariant): OleVariant; safecall;

function AS_DataRequest(const ProviderName: WideString;

    Data: OleVariant): OleVariant; safecall;

function AS_GetProviderNames: OleVariant; safecall;

function AS_GetParams(const ProviderName: WideString;

var OwnerData: OleVariant): OleVariant; safecall;

function  AS_RowRequest(const ProviderName: WideString;

    Row: OleVariant; RequestType: Integer;

var OwnerData: OleVariant): OleVariant; safecall;

procedure AS_Execute(const ProviderName: WideString;

const CommandText: WideString; var Params: OleVariant;

var OwnerData: OleVariant); safecall;

end;

Figure 2: We now use the IAppServer interface instead of IProvider.

Practically speaking, this means that instead of right-clicking a TDatasetProvider component and selecting Export Provider from Data Module, you set TDatasetProvider.Exported=True for any DatasetProvider you wish to export. (More information on why I mention TDatasetProvider will follow shortly.) This also means you must bind a TDataset to a TDatasetProvider and export the TDatasetProvider in MIDAS 3. While this is different from past versions, it’s a very minor price to pay, given the increased flexibility a TDatasetProvider component gives you as opposed to directly exporting a TDBDataset. The RDM’s IAppServer interface is then used to scan the list of providers that are exported in this manner. So there are no visible changes on the client side to forge the link. You still set ClientDataset.RemoteServer and ClientDataset.ProviderName as you always have.

The down side of the new approach is that you must convert all your client applications to get away from IProvider and use the new IAppServer interface instead. This means calls to ClientDataset.Provider.DataRequest and ClientDataset.Provider.ApplyUpdates — to name two — must be rewritten to access the new ClientDataset.AppServer property. A moderate application will take 15-30 minutes to convert once you understand what you need to do and you’ve done it a few times. Fortunately, the online help is superb in telling you exactly what you need to do to make this conversion a successful one.

No More TProvider

In addition to removing the IProvider interface, the TProvider component has been removed. The reason for this change is the introduction of a new interface, the IProviderSupport interface. All the things a provider must do to be considered a MIDAS provider has been abstracted into this interface. As far as MIDAS is concerned, there is no longer a delineation between TDataset and TDBDataset. The onus is now on the TDataset descendant to tell MIDAS what to do when asked, as opposed to the provider trying to provide least-common-denominator functionality for all datasets. It’s for this reason that TDatasetProvider is the new way to do things in MIDAS. TProvider still exists for backwards compatibility, but is not installed on the Component palette.

Any TDataset descendant that wants to participate as a MIDAS provider must implement the appropriate methods of IProviderSupport. For example, while TDataset is the first component in the hierarchy to implement IProviderSupport, it provides mainly blank method implementations. As you walk down the hierarchy to TADODataset, TBDEDataset, TDBDataset, and TQuery/TTable, further refinements to the implementation take place. When the Provider needs to get information or apply updates, it uses the IProviderSupport interface implementation for the TDataset that it is bound to.

TSocketConnection Changes

TSocketConnection has also undergone some surgery for this release. A new property, SupportCallbacks, has been added to the TSocketConnection. If this property is set to False, callbacks will not be supported in your application. Therefore, if you’re not using callbacks, set this property to False. The advantage of doing so means you only need Winsock 1 to deploy your client application. Win95 machines don’t come with WinSock 2, so it was one more impediment to a smooth client-side deployment. In addition, performance will be better if you disable callback functionality.

Another change to TSocketConnection (and TWebConnection, which I’ll explain later) is made on the security front. In the past, TSocketConnection allowed you to run any automation object on the server. This hole has been plugged by requiring you to override the UpdateRegistry method in the RDM. In this method, you can call one of several helper functions that will mark your application server as registered. See Figure 3 for a sample listing of one such UpdateRegistry method. All the helper functions will be explained throughout the course of this article.

class procedure TMyRDM.UpdateRegistry(Register: Boolean;

const ClassID, ProgID: string);

begin

if Register then

begin

inherited UpdateRegistry(Register, ClassID, ProgID);

      EnableSocketTransport(ClassID);

      EnableWebTransport(ClassID);

end

else

begin

      DisableSocketTransport(ClassID);

      DisableWebTransport(ClassID);

inherited UpdateRegistry(Register, ClassID, ProgID);

end;

end;

Figure 3: A sample listing of the UpdateRegistry method.

The call to EnableSocketTransport simply places a registry entry under the application server’s CLSID. This entry signifies that the application is available for running. One more option for you is the global override switch found in SCKTSRVR. If you uncheck the Connections | Registered Objects menu option, you eliminate the security check on the server.

RDM Pooling

RDMs are firmly rooted in COM. Because of this, the two main ways to create RDMs is to use DCOM’s instancing options of either ciMultiInstance or ciSingleInstance. ciMultiInstance RDMs spawn one process for all clients to use. Each client will have their own RDM, but because the application is a single-threaded application, only one RDM can be working at a time. This leads to unacceptable delays in a multi-user setting.

ciSingleInstance RDMs will spawn one process per client, thus taking more memory and resources. This option is only viable if you have a few clients connecting. The reason for this is that if you’re using the BDE, each process counts as one application that uses the BDE. The BDE has a limitation of 48 processes per machine. Therefore, the 49th client will not be able to spawn the 49th copy of the server.

The solution to these two sub-optimal choices prior to Delphi 5 was to use the TThreadedClassFactory to get the best of both worlds. One process would be spawned for all clients to share, avoiding the multiple-process problem of ciSingleInstance. Furthermore, each client would get its own RDM in a separate thread, thus avoiding the delay problems inherent in ciMultiInstance. Unfortunately, spawning unlimited threads could cause performance problems.

Delphi 5 solves all these problems by introducing object pooling for the RDM. To set up RDM pooling, you simply make a call to the RegisterPooled helper function in the overridden UpdateRegistry method of the RDM. You specify the maximum number of RDMs to create, and the amount of idle time before the RDM gets freed. If you’re using free threading in your RDM, you can also mark this RDM as a singleton. The declaration for RegisterPooled looks like this:

procedure RegisterPooled(const ClassID: string;

  Max, Timeout: Integer; Singleton: Boolean = False);

When you attempt to instantiate an RDM that is marked for pooling from the client, an intermediary gives out one of the free RDMs in the pool. If all RDMs are in use, you’ll get a “Server too busy” error message. No further work on your part is required to take advantage of RDM pooling.

At the time of this writing, RDM pooling was only available with TWebConnection. By the time Delphi 5 hits the shelves, TSocketConnection may support this feature as well.

Threaded Server Changes

Writing multi-threaded MIDAS application servers has been pretty easy since the introduction of TThreadedClassFactory. This class functions as a replacement to the standard TComponentFactory that Delphi surfaces for COM object creation. However, TThreadedClassFactory has not made it through a full QA cycle, and has been relegated to the DEMOS area of the product. There have even been some reports that this class factory replacement doesn’t work as well as it should when using multiple processors.

With MIDAS 3, writing a multi-threaded EXE is as easy as setting the CoInitFlags variable to COINIT_APARTMENTTHREADED. To make it even easier, the TComponentFactory now respects the ThreadingModel flag for EXEs, and will set CoInitFlags appropriately for you. For example, the following code in the initialization section of your RDM makes an application server multi-threaded:

initialization

  TComponentFactory.Create(ComServer, TEmpServer,

    Class_EmpServer, ciMultiInstance, tmApartment);

Remember that if you set this flag to make your application server multi-threaded, you must take care to protect global variables and GUI updates with thread-protection devices, such as Critical Sections or PostMessages. See the accompanying source code for the complete example of a multi-threaded server (see end of article for download details).

Figure 4 shows the result of this change: multiple threads are in use as multiple clients connect to the application server.

Figure 4: With the new threaded server changes, multiple threads are in use as multiple clients connect to the application server.

Introducing TWebConnection

You may have noticed this component in the table in Figure 1. I’m even willing to bet that a portion of you skipped right to this section to find out what TWebConnection is all about. Don’t worry. You won’t be disappointed.

TWebConnection is a TSocketConnection descendant that permits MIDAS traffic to be bundled into valid HTTP traffic, and thus use the most open port in the world, the HTTP port (default port 80). Actually, the component even supports SSL, so you can have secure communications. By doing this, all firewall issues are completely eliminated. After all, if a corporation doesn’t allow HTTP traffic in or out, there is nothing that can be done to communicate with them anyway.

This bit of magic is accomplished by providing an ISAPI extension that translates HTTP traffic into MIDAS traffic, and vice-versa. In this regard, the ISAPI DLL does the same work that scktsrvr does for socket connections. The ISAPI extension httpsrvr.dll needs to be placed in a directory capable of executing code. For example, with IIS4, the default location for this file would be in c:\inetpub\scripts. See Figure 5 for a screen shot of a typical setup using TWebConnection.

Figure 5: A typical setup using TWebConnection.

Another benefit of using HTTP for your transport is that an operating system like Windows NT Enterprise allows you to cluster servers. This provides true load balancing and fault tolerance for your application server. For more information about clustering, see http://www.microsoft.com/ntserver/ntserverenterprise/exec/overview/clustering.

The limitations of using TWebConnection are fairly trivial, and well worth any concession in order to have more clients capable of reaching your application server. The limitations are that you must install wininet.dll on the client, and no callbacks are available when using TWebConnection. In addition, you must register the application server with the utility function EnableWebTransport in an overridden UpdateRegistry method. Refer back to Figure 3 for a sample listing of this method.

Web MIDAS Client

One of the more innovative features in Delphi 5 is the Web MIDAS client. Using the components found on the WebMidas tab of the Component palette, you can create a browser-only front end to your MIDAS application servers. This bit of magic is accomplished by using a few complementary technologies: MIDAS, XML, JavaScript, and the Delphi WebBroker.

To build a complete Web client, you must understand the WebBroker architecture. This article will not explore this topic in any detail, but rather provide an overview of the steps you need to take to build a rudimentary Web MIDAS client.

First, you need to create a Web server application by selecting File | New | WebServer. I’ll create an ISAPI application here, as the application will run under IIS4. Next, add a TDCOMConnection component to the WebModule and set the ServerName property. This will act as your conduit to the application server from the ISAPI application.

Next, place a TXMLBroker component on the WebModule and link it to the DCOMConnection component you just placed on the WebModule. In this regard, you can think of the XMLBroker component as the WebMidas equivalent of the TClientDataset. The main difference is that the XMLBroker uses XML data packets instead of the OleVariants used by ClientDatasets.

If you want to specify an HTML page that will be sent automatically after a successful ApplyUpdates, you can tie the XMLBroker.ResponseProducer to any PageProducer component. Reconciliation errors will be dealt with by specifying a PageProducer capable of dealing with these errors. A standard HTML page, ERR.HTML, is included in the <Delphi>\SOURCE\WEBMIDAS directory. It serves as a convenient starting place to deal with reconciliation errors, much like the RECERROR.PAS unit is the standard way to deal with reconciliation errors in Windows applications. We’ll ignore the other properties and events on the XMLBroker component.

Finally, place a MidasPageProducer component on the WebModule. This is the key component that will generate the HTML content that the browser will eventually see. Notice that the HTMLDoc property has a default template inside it already. The HTML tags will expand at design time, when you create the HTML content by using the Web Page Editor (available by double-clicking on TMidasPageProducer).

The main property to note on TMidasPageProducer is IncludePathURL. This is the property that will determine where the application will look for the JavaScript files that help produce the WebMidas application. You’ll need to deploy the *.js files found in <Delphi>\SOURCE\WEBMIDAS to a location accessible by your ISAPI application.

To customize the HTML that will display from your ISAPI application, press the New button in the Web Page Editor. For this sample, we’ll choose DataForm. While selecting DataForm in the tree view, press New again. You’ll be presented with a list of components that you can add based on your selection. Select DataGrid and DataNavigator to add these elements to your HTML page.

At this point, select the DataGrid element in the tree view and view the Object Inspector. If you set the XMLBroker property to the component you added above, you’ll see a grid display in the resulting HTML code below. Do the same thing for the DataNavigator and you’ll see a group of buttons that look like the DBNavigator. You can customize just about any attribute or element on the HTML page.

Figure 6 shows some of the key components and settings at design time.

Figure 6: Some of the key components and settings for the Web MIDAS client at design time.

Now that we’ve built the page, we need to send the resulting HTML page to a Web browser. The content is generated by creating a new WebModule Action (by double-clicking on the WebModule and pressing the Add button) and assigning the OnAction event to the following code:

procedure TWebModule1.WebModule1WebActionItem1Action(

  Sender: TObject; Request: TWebRequest;

  Response: TWebResponse; var Handled: Boolean);

begin

  Response.Content:=MidasPageProducer1.Content;

end;

Remember to set the PathInfo of the Action to something meaningful. This value will be used from the browser later to access the virtual page you just created.

To deploy this application, you need to compile the application and copy the resulting ISAPI DLL to a place that is capable of running scripting code, e.g. c:\inetpub\scripts. You also need to place your JavaScript files in the path mentioned in TMidasPageProducer.IncludePathURL. The only thing left is to fire up your browser and connect to the page, either directly or via a link on another HTML page. For example, typing http://dmisernt/scripts/empservxml.dll/MidasPageProducer1 would work if you specified TWebModule.Actions[0].PathInfo=MidasPageProducer1.

Figure 7 shows the Web MIDAS client in action.

Figure 7: The Web MIDAS client in action.

Miscellaneous Changes

Deployment of MIDAS applications has changed slightly with this release. Because IProvider is gone, there is no more need to deploy STDVCLnn.DLL. Also, DBCLIENT.DLL got a name change to MIDAS.DLL. Furthermore, COM is not being used to communicate with MIDAS.DLL. It’s still registered with COM, and COM objects exist inside it, but the mechanism to retrieve those objects makes calls to CoCreateInstance unnecessary. The real benefit to the new approach is that you should see no more “CoInitialize has not been called” errors.

One of the primary benefits of writing a multi-tier application is the ability to partition your business logic into the middle tier. In this tier, you can evaluate data as it comes from the client and is about to be written to the server. You may even end up changing some of the values in this tier to help the data conform to your business rules. Until MIDAS 3, you needed to devise your own mechanism to update the record, or refresh the entire ClientDataset — which could prove to be a very costly operation.

With the addition of the Provider.Options.poPropogateChanges property, any change you make on the application server will automatically be returned to the client and merged into the ClientDataset. Another common reason for using this feature will be when you use auto-incrementing keys, date-time stamps, or triggers to modify record values on the server.

The Provider.Options set added many other new values as well. Some of the more intriguing ones are the poDisbaleInserts, poDisableEdits, and poDisableDeletes elements. By setting these properties appropriately, you can prevent the ClientDataset that is linked to this TProvider from inserting, editing, or deleting records. This provides a new opportunity to centralize more business logic and security implementation details on the server, which is always a good thing when trying to make your client as thin as possible.

The ability to send dynamic SQL to the server is easier than ever. Using the ClientDataset.CommandText property, you can send a SQL statement from the client to the server. As long as you set Provider.Options.poAllowCommandText to True, this change will be accepted. The statement will be executed when you issue either an Open or Execute command. Execute is a new method that allows you to execute queries on the server that don’t generate a result set. Output parameters will be passed back, but no result set. This brings ClientDataset more in line with TQuery and TStoredProc.

Lastly, I would be remiss not to point out that this version of MIDAS appears to be far more stable than its predecessor, even while in pre-release. Bugs that were present in MIDAS 2 have been dealt with. And new features and functionality appear to have a very low defect rate. This optimal combination of new features and reduced bug count makes MIDAS 3 very impressive.

Conclusion

This article covered many of the new features that MIDAS 3 introduced. I’m sure there will be features in the release version that were not available in the version used to do this review. However, this is only the beginning. Each topic is begging for your detailed exploration to find out how you can best use MIDAS to create robust, efficient multi-tier applications. MIDAS has always been able to deliver data easily, and MIDAS 3 does just that — it delivers.

The files referenced in this article are available for download.

Dan Miser is a long-time Delphi programmer and consultant, specializing in multi-tier application design using MIDAS. He is active in the Borland newsgroups, where he serves as a proud member of TeamB (http://www.teamb.com). Dan also finds time to write for Delphi Informant and speak at Borland conferences. You can visit his Web site at http://www.execpc.com/~dmiser, or contact him at mailto:dmiser@execpc.com.

Delphi 5 Enterprise Edition brings with it MIDAS 3, which introduces several new components, properties, and events. Mr Miser explains the new features, and even provides some sample code.

Mr Midas explains the new features of MIDAS 3.