Making an Open-Source Web Application at Home in the Microsoft World

Writing a Plugin for Microsoft Internet Information Server, by Joseph A. Bank (jbank@arsdigita.com)

Submitted on: 2000-10-20
Last updated: 2000-10-20

ArsDigita : ArsDigita Systems Journal : One article

Running an open-source Web application in a Microsoft world doesn't seem like a natural fit. However, ignoring the Microsoft world neglects a sizeable user community. With a couple of weeks of effort, we were able to get our open-source Web Application running on Windows 2000 using Microsoft's Internet Information Server (IIS).

Our Problem: How do I get ACS to run on IIS?

ArsDigita has developed a large body of source code implementing a proven data model and workflow called the ArsDigita Community System (ACS). IIS is one of the most popular Web servers. IIS has both a large entrenched user base and an assortment of software support. The basic problem is then simply: How can I get these two useful tools, ACS and IIS, to work together?

The two most obvious solutions have some fundamental problems:

• Solution 1: Give up IIS and use AOLserver. ACS using AOLserver is already supported on Windows 2000 (see http:/www.arsdigita.com/doc/win2k-installation).
• Solution 2: Port ACS source code to use IIS's APIs.

Solution 1 will often be unacceptable for an organization already using IIS. For someone starting from scratch who wants to run ACS on Windows 2000, using AOLserver directly would be the recommended course of action. However, for an organization that is depending on an installed base of Active Server Pages in Visual Basic or a commercial add-on to IIS, porting to AOLserver is not viable. As for Solution 2, porting ACS would be a large effort. ACS was developed using AOLserver, a free open-source application server with a rich set of APIs. The APIs provided by AOLserver are not directly supported by IIS. Porting ACS to the standard API's supported by IIS such as Active Server Pages in Visual Basic would be both a long and difficult process. Additionally, one of the selling points of ACS is that ArsDigita's 150+ developers release an improved version every 6-8 weeks. The developers are charged with building the best solutions to the world's collaboration problems. We do not want to distract them with having to maintain a VB version in addition to the two current versions (AOLserver TCL API and 100% Java).

Our Choice: Use both.

There is a way around the problems associated with the other two solutions: IIS supports an extension API called Internet Server API (ISAPI).

Microsoft supports two ways of extending IIS: ISAPI extensions and ISAPI filters. An ISAPI extension is conceptually similar to a traditional CGI program. The ISAPI extension is given information about the request from IIS and returns a Web page to IIS. Unlike CGI, an ISAPI extension is loaded once and has access to a more powerful set of APIs. An ISAPI filter does not have a CGI equivalent. An ISAPI filter is given access to both sides of the communication stream and can modify it at various points. For example, an ISAPI filter can provide encryption by modifying the input and output stream. Both ISAPI filters and ISAPI extensions are implemented with dynamically loaded libraries (dll's).

AOLserver is an extensible multi-threaded Application Server. It's modular architecture supports various dynamically loaded extensions. In fact, the standard Web server communication support is handled by an easily replaced module. This architecture allows new communication modules to be seamlessly added without any modification to the core AOLserver code or the code using the AOLserver APIs.

Using ISAPI solves our problem. We can use ISAPI dll's to run AOLserver "inside" of IIS and thus get the full set of AOLserver APIs without requiring any changes in ACS. Since AOLserver is open-source, we can modify AOLserver to create an ISAPI dll that uses IIS for the communication and AOLserver to produce content.

Architecture

The flexibility of ISAPI provides a number of different ways we could implement our basic idea. In this section, we describe the method we choose and then discuss the alternatives.

We choose to implement the AOLserver API by combining both an ISAPI extension and an ISAPI filter. Most of the work is done by the ISAPI extension, which implements all of the AOLserver functionality. The ISAPI filter is used only for mapping URLs.

Processes involved in running ACS using IIS.

An example helps to illustrate this design. Without using the ISAPI filter, a url might look like

   http://acs-on-iis.arsdigita.com/aolserver/nsisapi_extension.dll?/bboard

This request has the effect of sending the "/bboard" argument to AOLserver as if AOLserver received an HTTP request for "/bboard".

The ISAPI filter gives a flexible way to remap URL's to use the ISAPI extension. For example, on the acs-on-iis site, the url

   http://acs-on-iis.arsdigita.com/bboard

is internally rewritten to

   http://acs-on-iis.arsdigita.com/aolserver/nsisapi_extension.dll?/bboard

This change causes IIS to use the AOLserver ISAPI extension to handle the request.

The table below describes in more detail the steps involved in handling an HTTP request using this architecture.

Steps Involved in Handling a Sample Request
Note that all of the components are part of the same Windows process.

Component	Thread	Description
IIS	T1	HTTP request for "/bboard" is received by IIS.
ISAPI Filter	T1	IIS invokes the ISAPI filters on the request. As a result the request is changed from "/bboard" to "/aolserver/isapi_extension.dll?/bboard".
IIS	T1	IIS dispatches on the request, invoking the ISAPI extension. If the extension is not yet initialized, IIS invokes the ISAPI extension's "initialize_extension" function. For the AOLserver ISAPI extension, this runs AOLserver's initialization process. IIS invokes the ISAPI extension's "HttpExtensionProc" with an object describing the request.
ISAPI Extension	T1	The ISAPI extension queues a new IIS_Connection object representing the request into an AOLserver request queue.
ISAPI Extension	T1	The ISAPI extension returns "HSE_STATUS_PENDING" indicating that the request will be handled asynchronously.
AOLserver	T2	An AOLserver thread running in the ISAPI extension takes the request off of the queue and starts to process it.
AOLserver	T2	AOLserver executes the Tcl script found in /bboard/index.tcl, making various queries to an Oracle Database.
AOLserver	T2	AOLserver returns data for the page by writing to the IIS_Connection object.
ISAPI Extension	T2	The IIS_Connection object writes to the IIS request object.
IIS	T2	IIS sends data to the requesting client.
AOLserver	T2	AOLserver calls close on the Connection object.
ISAPI Extension	T2	The IIS_Connection object signals to IIS that the request object can be freed.
IIS	T2	IIS closes the connection to the client and frees the request object.

The current version of the ISAPI filter shows an example of the type of flexibility this architecture provides. It supports an option to use the hostname of the URL to determine whether or not to use the AOLserver ISAPI extension. For example, by configuring a single IIS machine respond to both "acs-on-iis.arsdigita.com" and "normal-iis.arsdigita.com", any requests for "acs-on-iis.arsdigita.com" will be redirected to use the AOLserver extension and any requests for "normal-iis.arsdigita.com" will not be redirected. Further extensions to this architecture would be very straightforward. For example, to use the bboard module of the ACS, you could modify the ISAPI filter to redirect urls starting with /bboard to use the AOLserver ISAPI extension.

Note: A similar architecture is used by the Tomcat open-source Java Servlet Engine plugin for IIS.

Other Architecture Options

Since we'd like to be able to support arbitrary URL's, using an ISAPI extension alone was not an option. When using an ISAPI extension alone, all requests would look like .../nsisapi_extension.dll?..., which is unacceptable because it would require changing the HREF links in all ACS pages. We considered some different ways to divide the work between an ISAPI filter and extension, and we considered how to pass data between the two, but those design choices were fairly minor. The other main option that we considered was to use an ISAPI filter alone and drop the ISAPI extension.

There are a couple of different reasons that we felt a design that used an ISAPI filter alone was inferior:

1. IIS 5.0 supports two different models of execution for ISAPI extensions. An ISAPI extension can run in either the same address space as IIS, or in its own address space. An ISAPI filter always runs in the same address space as IIS. Running in the same address space provides higher performance, but means that any errors in the extension can and do crash IIS. The ability to run in a different address space during development as well as the flexibility to minimize the effect of production bugs was very compelling. If performance became critical, an ISAPI extension supported both options.
2. While the APIs provided to an ISAPI filter are clearly powerful enough to implement the required functionality, the entire input stream is available and the output can be modified. However, the API for ISAPI extensions was sufficient and was a better fit for our purposes. For example, the ISAPI extension has access to the fully processed HTTP headers, while the ISAPI filter simply has access to the request input stream.

Required Tools

• Microsoft Windows 2000 Professional or Server.
• Microsoft Visual C++.

Development of the AOLserver ISAPI filter and extension for IIS required a small set of tools. First, Microsoft Windows 2000 Professional or Server is needed. IIS 5.0 is bundled with both versions of Windows. The AOLserver source distribution includes a configuration for building AOLserver using Microsoft Visual C++, so we used the integrated compiler and debugger of Visual C++. Visual C++ also comes with Microsoft Developer Network CDs, which includes API documentation and examples. For my own development, I also use Emacs and various Cygnus "cygwin" tools. It would probably work to use the gcc compiler and gdb debugger instead of Visual C++.

What Happened

The overall project of getting the ArsDigita Community System to run under IIS took two calendar weeks, of which the actual coding time was two days. A majority of the time was spent in setting up a development environment, learning the AOLserver code, and reading the ISAPI specification.

The project was divided into four major steps:

1. Get AOLserver to compile and load as an ISAPI dll. In this initial phase, the goal was simply to get AOLserver compiled as an ISAPI dll and loaded by IIS. IIS and the ISAPI dll did not communicate in a functional way at this stage; requests from IIS to the ISAPI dll were ignored. AOLserver was configured to listen on a different socket port, so that it could handle HTTP requests on its own to test its functionality. This phase actually turned out to be fairly significant since AOLserver has a complex initialization process that includes dynamically loading other dll's of its own. The modifications here were very small, but setting up the environment and learning to debug a running ISAPI extension took some time.
2. Write the "socket driver" code to get the AOLserver ISAPI extension to handle some simple page requests from IIS. AOLserver provides a modular "socket driver" specification for handling requests from different types of input, i.e. standard sockets, SSL, files, etc. The socket driver specifies a simple set of communication functions such as read, write, and accept. These calls were pretty straightforward to map into the ISAPI specification, producing a couple of pages of code, which represented most of the coding in the project. At this point, some simple pages using AOLserver APIs could be served through the extension.
3. Write ISAPI filter code to do the redirections to the ISAPI extension. This stage was also very straightforward, corresponding closely to standard ISAPI filter examples included with Visual C++. After verifying that it was easy to add different types of mappings, we chose a simple scheme of using hostnames.
4. Get it to work. The rest of the time was spent setting up a full ACS configuration, tracking down and fixing various problems, and cleaning up the code and configuration.

Future Work

Areas for improvement include the following:

1. Use IIS's asynchronous writes. One of the limitations of AOLserver is that it does not support asynchronous writes. Thus, a slow client can hold onto a thread, tying up server resources. Since IIS supports asynchronous writes, the ISAPI extension could take advantage of this, freeing up thread resources and potentially providing a performance improvement.
2. Enhance the ISAPI filter. In an environment integrating ACS/AOLserver code and ASP/VB code, the filter needs to have a more complex configuration indicating when it should use IIS's built-in application server and when it should use the AOLserver ISAPI extension.
3. Improve installation, setup and shutdown.

Summary

Developing an ISAPI filter and ISAPI extension was a quick and effective way of integrating an open source application server with IIS. You can download both a binary and source version of the AOLserver ISAPI Filter from http://www.arsdigita.com/download. Even if you don't want to use AOLserver, it serves as a good example of how ISAPI works.

As far as I can tell from this document, the exact same result could have been achieved in less than an hour by setting up the load balancer to forward certain URLs to different servers. This too would give you both ASP and ACS content on the same hostname.

-- Bas Scheffers, December 29, 2001