ArsDigita Archives
 
 
   
 
spacer

Download Design Documentation

by Alan Dickens

I. Essentials

Terminology: for purposes of this document, a "downloadable" is a collection of one or more files, together which constitute the different versions of a file. For example, all the different releases of the ArsDigita Community System, when taken together, consist of a downloadable.

II. Introduction

The Download application provides a system to facilitate user downloads of versioned releases of a file. Each downloadable group of files may be of any format or content, but typically are software or other data.

From the user's perspective the system offers a set of files available for download. These files are grouped according to download name - this name is the general name for one or more files that are different versions of the same file. For example, the name of the download may be "ArsDigita Community System", and the versions might include v.4.0b and v.3.4.6.

From an administrators perspective, the Download application offers the capability to make versions of a file available to users, indicate which version is the most current, to optionally require payment for the download of files, and to track (and if desired SPAM) users who have downloaded specific files.

III. Historical Considerations

The Download application was originally developed by Philip Greenspun, and is one of the older applications in the ACS. It was substantially enhanced in fall of 1999 by Sarah Ahmed, and in spring of 2000 ownership passed to Alan Dickens.

IV. Comptetitive Analysis

Many existing sites implement similar functionality with a FTP (File Transfer Protocol) server. FTP provides a convenient mechanism for transfering files, but provides none of the versioning, descriptive, adminstrative, or reporting features present in the Download application.

V. Design Tradeoffs

When designing the download system, the decision was made not to store the files in the database (as lobs) but rather to store them using the file system of the web server. This has various implications for server configuration, system administration, and backup procedures. Server configuration must account for the size of files anticipated to be made available as part of the download module, and hard drive space must be made available to accomodate the files. In a multi-disk server, there may be contention issues regarding which disk(s) the downloadable file are stored on; resolution of these issues is beyond the scope of this document. From a system administration perspective, the directories in which the downloadable files are to be written must be accessible to the web server. From a data safety perspective, backup procedures must account for the presence of this important content not in the database but rather as part of the file system. Backup procedures must be designed to account for the files in a manner consistent with the frequency that new versions are made available on the server.

VI. Data Model Discussion

This system consists of several tables and a pair of PL/SQL functions used to determine access permissions.

The downloads table is the master table that stores information about each set of downloadable files. In addition to the user visible name, it stores the directory name where the different versions of the downloadable are kept, and permissioning regarding who (which groups) of users are allowed to access the file. It also keeps a human readable version of the downloadable name. Finally, it also keeps various auditing information about the downloadable.

The download_versions table keeps the individual versions of each download. Each version has a seperate id, version number and description, as well as the filename used to access the version.

The download_rules table contains rules about which classes of users are allowed to see and download which files. This is quite useful to (for example) allow all paid users of version 1.2 to download 1.2.1 for free.

Next, a pair of PL/SQL functions to determine who can see each file, and also who is allowed to download each one. The function download_viewable_p is used to determine if a user can view a file, while the function download_authorized_p determines if a user is authorized to download the file.

Finally, the download_log table tracks who has downloaded which versions of which files, including their reasons (optional) for downloading the file.

VII. Legal Transactions

From the Maintainer's administrative pages at /download/admin or /groups/admin//download/ the download maintainer can

  • View an individually selectable list of all downloadables
  • Add a new downloadable to the system
  • View documentation on the download system
  • View the user pages for the download system

After selecting a downloadable, the maintainer can

  • View information about that downloadable, including:
    • Download Name
    • Directory Name
    • Description
  • Edit the above mentioned information about the downloadable
  • Remove the downloadable
  • Upload a new version of the file to the downloadable
  • A selectable list (divided by version status) of all versions of the file that compose the downloadable
  • Add Rules regarding which users can see the various versions of the downloadable, and costs (if any) associated with downloading those version
  • View a history of downloads of various versions of the downloadable

When uploading a new version into the downloadable, the maintainer must supply:

  • The file to the uploaded
  • The name given to the file in the downloadable
  • The version of the new file in the downloadable
  • The status of the version:
    • Promote - offer first to the user
    • Offer If Asked - offer only if the user asks to see older versions
    • Removed - do not offer to the user (primarily used for obsolete versions)
  • The date that the version should be made available
  • A description (optionally containing HTML formatting tags) of the version

When viewing an individual version of the file in the downloadable, the maintainer can:

  • View various infomration associated with that version
  • Edit the information associated with that version (including updating the status)
  • Remove that version (strongly discouraged except for really truly ancient versions of the file)
  • View a download history of the file

When adding a rule regarding the downloadable, the maintainer can set:

  • Which version of the downloadable the rule applies to (or all versions)
  • Whether the affected version(s) are visible to all users or just a group of uers (e.g. only registered users)
  • Whether the affected version(s) are available to all users or just a group of uers (e.g. only registered users)
  • The price (if any) and currency associated with accessing the version(s) in question

When viewing the Download History, the maintainer may be viewing the history for the entire downloadable, or just for a single version of the file. This history shows:

  • The name of the downloading user
  • The email address of the downloading user
  • The date of the download
  • The IP address from which the download occured
  • The version downloaded
  • The current status of that version
  • A link to the reason for the download
  • A link to remove the log of that download
From this page, the maintainer can:
  • SPAM (send mail to) all users reported on the page
  • View all the Download Reasons given by those users

VIII. API

The Download application provides the following scripting (TCL) procedures which developers might find useful:

download_serve { conn context }: give the conn and context, this procedure attempts to:

  1. Compose the true name of the file requested
  2. If no version id is specified, offer the most recent version
  3. Determine if the user has permission to download the file
  4. If so, return the file to the user, logging the event

download_admin_authorize { download_id }: given download_id, this procedure will check whether the user has administration rights over this download. if download doesn't exist page is served to the user informing him that the download doesn't exist. If successfull it will return user_id of the administrator.

download_version_admin_authorize { version_id }: given version_id, this procedure will check whether the user has administration rights over this download. if download doesn't exist page is served to the user informing him that the download doesn't exist. If successfull it will return user_id of administrator.

proc_doc download_version_authorize { version_id }: given version_id, this procedure will check whether the user has visibility rights over this download. if download doesn't exist page is served to the user informing him that the download doesn't exist. If successfull it will return user_id of the user.

proc_doc download_mkdir { dirname }: Ensures that \$dirname exists. Won't cause an error if the directory is already there. Better than the stardard mkdir because it will make all the directories leading up to \$dirname". Used when creating a new downloadable to create the directory in which the download files will reside.

download_version_delete { version_id }: deletes the download version file from file storage and related data from the database .

download_date_form_check { form date_name }: checks that date_name is a valid date. Returns a list of date_name exception_count, exception_text. Sets date_name to a YYYY-MM-DD format.

IX. User Interface

The Site-wide Administrator Interface

When the administrator decides to add a new downloadable file to the system, first the download directory must be created. In the current version of the module, this takes the form of the "Add New Download" function on the admin pages. Doing this function only creates the directory in which the downloadable files are stored.

Actually placing the files in that directory is a second and seperate step. To upload a file to the server, select the directory name from the admin screens and then choose "Upload New Version" command. This allows the administrator to place a file on the server that is downloadable by users.

The User Interface

To regular users, the downloadable file is visible based on the assigned name assigned when the download directory was created, combined with the version number. This isn't always the most obvious presentation of download filename, and will likely be customized by applications that offer other than software releases.

Downloads with a status of "promote" are offered on the top-level /download/ page. Downloads with a status of "offer_if_asked" are available on a drill-down page of "/download/one" where multiple versions of the same download may be offered. Downloads with a status of "removed" are only shown to the site or group administrator who has the option of changing their status or looking at an old version.

Users can see all files available for download (even before registration). However, upon actual download request of a file, the system checks the user's registration status and the visibility (e.g. "all", "registered_users", or "group_members") of the specific file and permits download accordingly (e.g. non_registered users are only allowed to anonymously download files with visibility="all", he is required to login otherwise).

X. Configuration/Parameters

The directory into which the /download/ directory is placed (generally /web/server_name/) must be write accessible to nsadmin. This can either be accomplished by making the owner nsadmin or changing the permissions so that nsadmin has write privileges. Either way, if nsadmin can't write to this directory, an error indicating that "Folder could not be created" will result. If the intended use of the Download Application involves documents that can be opened by a program on the receiving machine (e.g. files of type .jpg, .doc, .pdf, etc.), then the AOLserver's .ini file (not the ACS .ini file) must be modified to make AOLServer aware of these file types. This entry in the .ini file takes the form: 

[ns/server/servername/mimetypes]
.doc=application/msword
.ppt=application/powerpoint
.xls=application/excel

XI. Acceptance Test

You should test adding new downloads, upload versions of existing downloads, adding rules to all versions/specific version of a download, edit/remove download data, download a file, view download history.

Suggested Steps:

  • Add a new download
  • Upload a new version of the download
  • Add a rule to the download
  • Edit the download data
  • Go to user pages and download the version
  • Go back to admin pages and view the download history
  • Remove the version
  • Remove the whole download

XII. Future Improvements

  • Although the data model supports "purchaser"/ "previous purchaser" user visibility, the functionality of charging for access to a downloadable file has not been implemented. It could be tied to a future version of the ecommerce application or be standalone.
  • Enhanced statistics and demographics: instead of just asking for a simple comment (on why the user is downloading), instead ask a set of survey questions.
  • Enhanced reporting: the ability to view downloads from [date] to [date], a breakdown of the total number of downloads by month, group downloads by domain name, and be able to view some results graphically.
  • At-a-Glance reporting: at the download/admin pages, each downloadable could show the total number of downloads, the number in the last month, and the number in the last week. These numbers should be drill downable to be able to see more detail on which versions were downloaded.

XIII. Authors

adickens@arsdigita.com
spacer