Building Documents in ACS

part of ACS Core Architecture, by Philip Greenspun and Jon Salz on 22 May 2000

Tcl procedures: /packages/acs-core/document-procs

This is an API for programmers writing scripts in the ArsDigita Community System that return documents (a degenerate form of which is a Web page) to Web clients.

The Big Picture

Standard AOLserver programming, like CGI scripting before it, had the programmer directly writing bytes to a client browser connection. Thus from 1995 through 2000 an AOLserver programmer would build up a complete HTML page in a script and write the bytes of that page to the client either all at once with ns_return or incrementally with ns_write.

Problems with this standard old approach:

difficult to control style on a site-global basis via a master template
difficult to write a script that returns an XML document that is then rendered by a higher-level request processor (i.e., each individual script has to be aware of all possible document types that might be required by client, e.g., HTML, WML, XML, etc.)
easy for novice programmer to create server performance problems by calling API procedures that block in the TCP stack while holding a database handle from the pool (i.e., a script could be waiting for a client on a slow modem to accept some packets while holding a database connection from a limited pool)

As of ACS 3.3, new modules and scripts can be constructed to build documents rather than pages. A document is a data structure containing a body and a series of attached properties (e.g., title). Once a document is built by a script, the request processor takes over and renders the document into something which the client can understand, e.g., an HTML page.

How To Use It, In Twenty-Five Words Or Less

Use doc_body_append instead of ns_write.
Use doc_set_property to set the title and navbar.
Don't write page headers or footers; leave that to the master template.

Using the Document API: An Example

The Pre-3.3 Way: Writing to the Connection

Consider the following hypothetical old-style Tcl page (call it user-list) that writes out a list of names of registered users:

ReturnHeaders "text/html"

ns_write "[ad_header "User List"]
<h2>User List</h2>

[ad_context_bar_ws [list "index" "Users"] "User List"]
<hr>

<ul>
"

set user_list ""

set selection [ns_db select $db "
    select first_names, last_name from users order by upper(last_name)
"]
while { [ns_db getrow $db $selection] } {
    set_variables_after_query
    append user_list "<li>$first_names $last_name\n"
}
ns_db releasehandle

ns_write "$user_list</ul>

<hr>
[ad_footer]
"

This is all well and good, but what if we decided we wanted to make that title appear in a sans-serif font on every page in the site, and have the context bar appear in a right-aligned table next the body, and eliminate the <hr>s as well? We couldn't, without manually fixing every single file (or having Jin write a Perl script).

The Enlightened Way: Building a Document

For reasons like this it's a good idea to break each page into a set of separate pieces which a master template can piece together however it sees fit. For HTML pages in general, we identify three pieces of information we can split up easily:

The page's title.
The navigational bar.
The body of the page (everything between the <hr>s).

The document API allows us to output these pieces separately. user-list becomes:

doc_set_property title "User List"
doc_set_property navbar [list [list "index" "Users"] "User List"]

doc_body_append "<ul>\n"


set selection [ns_db select $db "
    select first_names, last_name from users order by upper(last_name)
"]
while { [ns_db getrow $db $selection] } {
    set_variables_after_query
    doc_body_append "<li>$first_names $last_name\n"
}

doc_body_append "</ul>\n"

# we can release the db handle anywhere in the script now; nothing
# gets written to the client until we return (or unless 
# doc_body_flush is called)
ns_db releasehandle

None of this actually writes to the connection. It calls a few magical APIs, doc_set_mime_type, doc_set_property, and doc_body_append, which construct a data structure. This data structure is then passed by the request processor to an ADP master template (usually called master.adp), which might look like:

<html>
  <head>
    <title><%= $title %></title>
  </head>
  <body bgcolor=white>
    <h2><%= $title %></h2>
    <%= [eval ad_context_bar_ws $navbar] %>
  <hr>
    <%= $body %>
  <hr>
    <address>jsalz@mit.edu</address>
</html>

Note that to refer to the document properties (title, navbar) we just use the usual ADP syntax for reading variables, e.g., <%= $title %> to read the title property. The same goes for the document body, which is read with <%= $body %>. The request processor locates the appropriate master template for a page as follows:

In the directory containing the file being delivered (e.g., user-list), look for a file called master.adp. If there's one there, use it. If not:
Look for master.adp in the parent directory of that directory. Keep traversing up the directory tree until a file called master.adp is found.
If no master.adp file is found anywhere in the main code tree, use the master.adp file in the templates directory (the site-wide default master template).

For example, if user-list is really /web/arsdigita/www/users/user-list, we'll check for a master template at

/web/arsdigita/www/users/master.adp
/web/arsdigita/www/master.adp
/web/arsdigita/templates/master.adp

This allows us to provide a default master template, but to override it for documents in specific parts of the site.

ADPs as Documents

You can also build a document in an ADP file, by enclosing the body in <ad-document> and </ad-document>, and using the <ad-property> tag to set properties. The following ADP file (user-list.adp) is equivalent to user-list.tcl above:

<ad-document>
  <ad-property name=title>User List</ad-property>
  <ad-property name=navbar>[list [list "index" "Users"] "User List"]</ad-property>

  <ul>

<%

set selection [ns_db select $db "
    select first_names, last_name from users order by upper(last_name)
"]
while { [ns_db getrow $db $selection] } {
    set_variables_after_query
    ns_adp_puts "<li>$first_names $last_name"
}
ns_db releasehandle
%>

  </ul>
</ad-document>

Complete Tcl API

doc_set_mime_type mime-type

Sets the MIME type for the current document to mime-type. This defaults to text/html;content-pane (which means that we should try to apply a master template), so you probably won't need to change it in most cases. In the rare case that you do need to write a page which shouldn't be generated through the master template (e.g., a differently formatted HTML page, or a text/plain page), you'd use this procedure, e.g.,

doc_set_mime_type "text/html"

doc_set_mime_type "text/plain"

Then you'd use doc_body_append to generate your document, and the request processor would just serve your document as is.

doc_set_property name value

Sets the document-level property named name to value.

doc_body_append string

Appends string to the document body.

doc_body_flush string

Writes out as much as possible to the client. This is a dangerous API call because the programmer runs the risk of tying up a database handle if he or she is not thoughtful. Does nothing if the mime type has not been set. Does nothing if the document being produced must be rendered via a master template.

What's the Point?

Many more interesting and important things (which we'll add for ACS 4.0) fit into this framework:

By adding datasources (really just single- or multi-row properties) we'll be able to generate documents programatically which are semantically equivalent to .spec files in Karl Goldstein's templating system (but generated with a Tcl API).
More specific types of data (e.g., ASJ articles, bulletin board topics, and even individual items in topics) will turn into documents which are individually templatized (allowing us complete separation of data and presentation).
The request processor will have a set of rules allowing it to automatically determine which components to invoke based on document types and request headers (e.g., user agent) so we can dynamically serve the right form of data for a particular client.

This API is part of a gradual move toward this model.

Under the Hood

The doc_* API calls store stuff in a global variable named doc_properties. After the abstract URL system sources a file, it checks to see if anything's been set in doc_properties. If so, it analyzes the MIME type in the document that's been returned, and invokes a template or returns the body, as appropriate. If not, it assumes the page did its own ns_write or ns_returning (and doesn't do anything special).

philg@mit.edu

jsalz@mit.edu