ACS 3.2 Developers' Guide to ACS 3.3 and Beyond

This document tries to answer the question, "What the heck have you done to my beloved ACS?" It briefly describes the changes made to ACS between versions 3.2.3 and version 3.3, from a developer's standpoint. It is intended to contain the minimal set of new information with which everyone needs to be familiar to develop for ACS 3.3.

Big Picture

In the past, ACS has been a toolkit held together by loose conventions. While this has worked well up to the present, ACS has matured to the point where it needs to be more heavily structured. One of our main goals for versions 3.3 and 4.0 is to add this structure - to develop valuable conventions and clean abstractions that will help preserve ACS's status as the best open-source toolkit around for developing purposeful online communities.

In a Nutshell

The filesystem is reorganized, and the bootstrapping process changed, due to package management features.
ACS startup isn't as simple as it used to be.
A single request processor (not an ad hoc series of filters and registered procedures) now handles every single request to the site. You must use ad_register_filter and ad_register_proc rather than ns_register_filter and ns_register_proc!
The Database Access API has been improved.
A new document API will be taking over ReturnHeaders/ns_write/ns_return shortly.

Note that aside from the ad_register_* changes, you are merely encouraged, not required, to use the new APIs. As of 4.0 (and all code integrated into ACS as packages) you will need to use the new APIs. The next few months will be a transitional period.

Package Management and the Filesystem

One of the most fundamental changes to ACS starting with version 3.3 is the introduction of package management, and the reorganization of the filesystem (see the ACS Package Manager documentation to learn why this is a good idea). Looking at the ACS 3.3 directory, you'll notice a new /packages directory (at the very top of the directory tree, right alongside /tcl and /www. The idea is to divide ACS into a series of functional components, "packages," with each package mapped to its own directory inside the /packages directory. The ACS core package (/packages/acs-core) provides the very basic functionality expected of an ACS system.

It is outside the scope of this document to describe how to develop a package (but if you're interested, see the APM documentation or the 5-Minute Guide to Packaging Your Module) We're not asking anyone to build packages yet - just keep developing as you always have been for now, in the same directories, but know that a lot of files you might expect to see in /tcl, /www/doc/sql, etc. have been moved inside the ACS core package (/packages/acs-core).

The package manager UI can be accessed at http://yourservername.com/admin/apm/ (site-wide administrators only).

ACS Startup (Bootstrapping)

When AOLserver starts up, the first thing it does is to source every .tcl file in the /tcl directory. This still occurs, but very importantly, the 0-acs-init.tcl script is sourced first. The entire startup process (discussed in detail in the Bootstrapping) is:

AOLserver sources /tcl/0-acs-init.tcl.
0-acs-init.tcl sources /packages/acs-core/bootstrap.tcl.
bootstrap.tcl sources all *-procs.tcl files in the acs-core package.
bootstrap.tcl scan all directories inside the /packages directory, looking for package specification files (*.info files) which describe packages. If it finds any new ones, it loads them into the database.
bootstrap.tcl sources the *-procs.tcl files for all enabled packages.
bootstrap.tcl sources the *-init.tcl files for all enabled packages.
AOLserver continues to source files in the /tcl directory (i.e., every file after 0-acs-init.tcl in lexicographical order).

This more complicated process is necessary to support packages. The distinction between *-procs.tcl and *-init.tcl files is necessary to

prevent ordering conflicts between packages, where the initialization code for package A requires a procedure in package B which hasn't been defined yet (since B succeeds A in lexicographical order).
make it possible to implement a package manager feature similar to AOLserver 2's Reload Client Tcl. This requires a distinction between procedures to reload (*-procs.tcl) and initialization code that should truly be called only once (*-init.tcl).

Request Processor

Pre-3.3

In versions of ACS before 3.2, it's very difficult to determine exactly which code is running when a particular request is executed. In general, all kinds of filters are executed, and then some registered procedure (maybe the abstract URL handler) is invoked (or, if abstract URLs are disabled, a file is served directly). There are several problems with this approach, most notably that:

It's difficult to deliver files which don't physically live underneath the page root (in the /www directory), as in the case of pages associated with packages.
It's very difficult to control the order in which filters are executed.
It's difficult to determine which code is executed for requests to which URLs.
If something breaks, it's very difficult to determine which filter is causing the problem.
Scoped requests need to be handled specially, with ns_register_procs for each possible URL prefix (/groups, /some-group-type, /some-group-name, etc.).
Filters and registered procedures are strictly URL-based, so they break under scoping (e.g., a procedure registered for /download/files/ won't work for requests under /groups/some-group-name/download/files/62/smug.jpg).

In 3.3 and Beyond

In ACS version 3.3, absolutely every request is served through a unified request processor (rpp_handler in /packages/acs-core/request-processor-procs.tcl), described in detail in the request processor documentation. We have replacement routines for ns_register_filter and ns_register_proc, called ad_register_filter and ad_register_proc respectively, which solve the first four problems:

The request processor is smart enough to look inside packages to deliver files in their www and admin-www directories.
ad_register_filter has a -priority flag. Filters with numerically lower priorities run before filters with numerically higher priorities.
There are nifty monitoring tools under http://yourservername.com/admin/monitoring/ which let you determine which filters and procs are executed when a particular URL is requested.
Procedures registered with the new API provide full stack traces in the log when something goes wrong.

We'll solve the latter two problems as soon as subcommunities are implemented for 4.0.

Important: You must use ad_register_filter and ad_register_proc rather than the corresponding ns_ calls. ns_register_filter and ns_register_proc are disabled in the release version of ACS 3.3.

Database Access API

Usage of ns_db is being phased out (although we're not deprecating ns_db yet - you can continue to use it). That's right, we're providing a new Database Access API which totally frees you from the responsibility of allocating database handles and passing them around to subroutines. For instance, instead of

set selection [ns_db select $db "select foo, bar from greeble"]
while { [ns_db getrow $db $selection] } {
    set_variables_after_query
    append output "<li>foo=$foo; bar=$bar\n"
}
ns_db releasehandle

you can now just write

db_foreach "select foo, bar from greeble" {
    append output "<li>foo=$foo; bar=$bar\n"
}
db_release_unused_handles

Not a database handle ($db) in sight; the database handles are managed and recycled for you. Don't worry, this actually does work (despite the many incredulous emails we've received): the entire package manager was written with it and works without a hitch.

See the Database Access API documentation for more information, including a discussion of why this is A Good Idea^TM.

Document API

Fare thee well, ReturnHeaders, ns_write, ns_return, and friends! From Building Documents in ACS (philg, jsalz):

Standard AOLserver programming, like CGI scripting before it, had the programmer directly writing bytes to a client browser connection. Thus from 1995 through 2000 an AOLserver programmer would build up a complete HTML page in a script and write the bytes of that page to the client either all at once with ns_return or incrementally with ns_write.
Problems with this standard old approach:

difficult to control style on a site-global basis via a master template
difficult to write a script that returns an XML document that is then rendered by a higher-level request processor (i.e., each individual script has to be aware of all possible document types that might be required by client, e.g., HTML, WML, XML, etc.)
easy for novice programmer to create server performance problems by calling API procedures that block in the TCP stack while holding a database handle from the pool (i.e., a script could be waiting for a client on a slow modem to accept some packets while holding a database connection from a limited pool)

The basic idea is that, in Tcl scripts, you now use doc_set_property to set the title and navigational context for a document, and doc_body_append (rather than ns_write) to build up the body for the page (leaving out the headers and footers) master template later builds a page for you, tacking on header, title, navbar, footer, etc. as appropriate in a customizable way.

Don't worry, you don't need to start using this right away. But for more details, see Building Documents in ACS.

jsalz@mit.edu