Ecommerce Module Design
by
Sarah Ewen
I. Essentials
|
/tcl/ecommerce-defs.tcl |
- Procedures related to credit card transactions:
|
/tcl/ecommerce-credit.tcl |
- Procedures related to sending emails:
|
/tcl/ecommerce-email.tcl |
- Procedures related to calculating total order prices:
|
/tcl/ecommerce-money-computations.tcl |
|
/tcl/ecommerce-scheduled-procs.tcl |
- Procedures related to order state changes:
|
/tcl/ecommerce-state-changes.tcl |
- Procedures related to page styles:
|
/tcl/ecommerce-styles.tcl |
- Procedures related to user activity tracking:
|
/tcl/ecommerce-user-contributions-summary.tcl |
|
/tcl/ecommerce-utilities.tcl |
- Procedures related to interface widgets:
|
/tcl/ecommerce-widgets.tcl |
II. Introduction
What the Ecommerce Module is For
The ecommerce module is designed to give a web site all the functionality that you would expect from any large web site that sells products to its online community (e.g., www.amazon.com). It is intended to provide all the utilities that an ecommerce web site is likely to need, including:
- Security
- User tracking
- Gift certificates
- Mailing lists
- An auditing system
- Order fulfillment
- Order tracking
- Special offers
- Online reporting
- User-based product rating & professional reviews
- Product association (linking)
- Pre-ordering
Additionally, it is intended to be used with the customer service module, which was written to provide all the customer service functionality that the ecommerce module requires, namely: customer issue and interaction tracking.
The ecommerce module is designed to allow the shop to operate in either of the following ways:
- The web site owner is the retailer
- All the orders made on the site are sent to one retailer
With (a lot of) customization, it can also be set up to handle ordering products from multiple retailers.
The next version will account for another way of doing business: having multiple retailers set up their own shop on your ecommerce site, a bit like store.yahoo.com.
What It Isn't For
At present, you can't allow retailers to build their own stores on your site, as mentioned above. Also, it is not designed for small sites: the ecommerce module is aimed at large online communities selling thousands of products. If you just want to sell a handful of items and not build a full online community, you're probably better off starting with ArsDigita Shoppe, downloadable from http://arsdigita.com/free-tools/shoppe.
If you want to sell conference or course registration, use the events module (www.arsdigita.com/events). This code has been used to process thousands of course and marathon registrations.
III. Historical Considerations
Most of the considerations that influenced the ecommerce module were associated with specifications for client work. When the ecommerce module was written, there were certain requirements that influenced the design that was adopted and the features that were included, such as having three levels of depth in the product category tree. (When you think about it, having more than three levels of product categories is going to make browsing tiresome for customers - going down, down, down again, down, then up, back up, up, down, oops wrong one! up again, down again...).
Take a look at "Design Decisions" for the details on how everything works, and why.
IV. Competitive Analysis
There are a number of products out there that provide roughly the same services as the ecommerce module. The most notable of these are:
- BroadVision
- BlueMartini
- Oracle iStore
- Akopia Tallyman
- Dynamo Commerce Server
Rather than stepping laboriously through each individual feature that both have to offer, the following attempts to highlight interesting contrasts between the ecommerce module and the other products.
BroadVision
| ArsDigita Ecommerce Module |
BroadVision |
Customers can comment on products
Admins approve/disapprove comments |
Threaded message boards |
| Customers can give products ratings |
System recommends which discussion boards user should join, based on history |
| Administrators submit professional review of products; choose whether to display them |
Can create on-the-fly product comparisons |
| Customers save, view and retrieve shopping carts |
Customers can save shopping lists as well as shopping carts |
| Admins can view all products in one list; users can view lists of products per category or use a search |
Customers can view site wide lists of products |
| Admins can query for users based on their ACS history and send them gift certificates or emails |
Incentive programs can be loaded from exogenous business systems |
| Admins create user classes and can have `look and feel' templates and promotional offers associated with them |
"Contract Wizard" creates specific sets of products and prices for each company that accesses the web site. |
| Page templates can be created & saved, and then associated with user classes or products |
Provides content scheduling |
| Templates are written in HTML and can have ADP code; not compulsory but allows for very flexible and customized pages. No need to muck about in Tcl or SQL. |
Integrated with DreamWeaver; no need to understand technical underpinnings |
BroadVision vs ACS ecommerce module: Conclusion
- BroadVision is big. It supports some features in just about every area of ecommerce; marketing tools, publishing, customer service, order processing.
- Its size is also its weakness. It has made a lot of enemies through tendencies to create bloated URLs, having poor frame support, undocumented `features', and being quite painful to develop with. Take a look at http://www.peterme.com/bvsucks/.
- Despite it's size, it is still unable to address one of the most fundamentally powerful issues in ecommerce; really knowing your consumer. It might offer the ability to offer incentives to certain types of customer, but how will it get to know them? Spying on their shopping activities is a start, but reaches nowhere near the comprehensive profile building that the ecommerce module's tight integration as part of the ACS can achieve.
BlueMartini
| ArsDigita Ecommerce Module |
BlueMartini |
| Integrated into the ArsDigita Community System |
13 modules offering different aspects of ecommerce |
| Can take advantage of CRM module and the customer service module (specially developed for the ecommerce module) in the ACS |
Very focussed on customer management and customer targeting; extensive utilities |
| Customer service module developed for tracking issues customers raise and all interactions with customer services until they are resolved |
Call center for taking orders and managing them. |
| Collects information from interaction with community web site |
Collects information from users' purchasing and browsing history; `customer event stream behavior' |
| Use of ACS bulletin boards, discussion forums, community services |
Facility for customer collaboration whilst browsing products, real time chat |
| Admins save templates for use at any time, new templates can be created based on existing ones |
Content management versioning system |
| Admins have access to full searching capabilities of ACS; graphing module can be utilized to produce graphical representations of the results with some programming |
Data mining module offers multidimensional graphing of mining results |
| Admins create classes of users that they wish to define; users can be given the right to add themselves to these classes directly |
Personalization implemented through `business rules' that can be generated using a `rules induction' algorithm |
BlueMartini vs ACS ecommerce module: Conclusion
- BlueMartini is well structured; so is the ACS. They both adopt the modular approach. Both systems' product management systems are comparable, except that BlueMartini offers the ability to handle multiple units of currency simultaneously.
- BlueMartini offers a lot of good content management features such as a built-in thesaurus for null searches, internationalization, full text and parametric searches (also provided by the ArsDigita ecommerce module), but again loses out on being limited to ecommerce only. The `customer event stream behavior' can track order histories, billing preferences, gift registries, but so can the ACS, and then some.
- The ACS ecommerce module offers all the same personalization features as BlueMartini, but as yet offers nothing really comparable to the `customer collaboration' feature, although the ACS does offer extensive community services there is nothing quite so real time available.
- The data warehousing services are also comparable. However BlueMartini bundle thirty reports with the warehousing module; the ecommerce module instead offers set reporting features rather than customized reports.
- BlueMartini offers a call center, which compares to the orders section of the ecommerce module combined with the customer service module. BlueMartini does not appear to offer distinct support for customer issues and interactions, though.
Oracle iStore
| ArsDigita Ecommerce Module |
Oracle iStore |
| Integrated into the ArsDigita Community System; offers customer service and order management as one package |
Integrates with Oracle ERP applications and the Oracle Call Center & telephony suite for order management and extended customer care |
| Admin-defined user classes that can be associated with product look and feel templates, and promotions |
Personalization; merchant-defined options |
| Customers have to contact customer service reps and talk to them nicely about canceling orders |
Customers can cancel orders online. |
Oracle iStore vs ACS ecommerce: Conclusion
- iStore and the ecommerce module have a great deal in common; the functionality of both is very similar, even more so because the ArsDigita Community System is based on the Oracle 8i RDBMS.
- Both systems audit all important data changes and store the information in an Oracle database.
- Both offer batch product data loading utilities, and some pre-defined reports on site usage and status.
- Oracle can tie in more of their products by offering `call me' buttons on the site, that customers can sue to request that a customer service representative should contact them by telephone for help regarding their order. This features is implemented using their Call Center & Telephony suite. Like the customer service module in the ACS, customer service reps can add comments to the record about the nature of the customer's inquiry.
- While Oracle offer integration into ERP applications, allowing features like real-time inventory checks, they don't have anything comparable to the ArsDigita Community System.
Akopia Tallyman
| ArsDigita Ecommerce Module |
Akopia Tallyman |
| Can include ADP and product variables in page templates |
Can include perl in page design |
| Provides sales, order fulfillment, customer service, reports |
Focuses on sales and fulfilling orders. No customer service. Very basic reports on orders and site traffic. |
| Customers can save, view & retrieve shopping carts |
Customers cannot save shopping carts |
| Look and feel of the site changes depending upon user classes that the user may belong to |
Site cannot be customized per user |
| Extensive documentation |
On-line help |
| Custom fields for each new product type |
All product fields are pre-defined |
Akopia Tallyman vs ACS ecommerce: Conclusion
- Akopia Tallyman is open source; this is a big plus for any business who is not willing to bank their future and success on that of another company. However it has limited reporting, personalization and product management functionality at present and no customer service features, which doesn't make it a very practical solution for large online communities.
- Every time that the site pages are redesigned, the site has to be regenerated, which will become rather tiresome when you are a large online ecommerce community selling thousands of products with potentially very many different product page designs.
- It does offer some friendly on-line help; the ecommerce module could use the ACS help module, or a few static pages could easily be created.
Dynamo Commerce System
| ArsDigita Ecommerce Module |
Dynamo Commerce System |
| Offers one default template and the ability to write more in HTML and ADP |
Offers pre-built templates with the system and a templating `wizard' for designing more, not a language. |
| Different users can be offered special promotions; these are offered when browsing the site. Customers have their own `Your Account' page which shows them their entire order history and all the mailing lists they are subscribed to. |
Customers can have their own home page with their recent order history, products they bought recently and special promotions. |
| Executes credit card authorization and billing in real time. |
Offers a highly configurable `order processing pipeline' which must interface with third party software to make payments. |
Dynamo Commerce System vs ACS ecommerce: Conclusion
- The ArsDigita ecommerce module offers all the functionality of the Dynamo Commerce System except for slight differences in the customer home page, and the page template building utility. The Dynamo Commerce System offers a much more less techy-oriented approach, templates are designed using a set of questions rather than allowing the administrator to write the new template themselves. This results in a slightly less flexible template design, but does reduce the amount of technical knowledge required to design a page template.
- The ecommerce module is open source and is able to process payment internally using CyberCash. This makes it a more attractive solution than relying on more closed-source third party software.
In summary
There are a small number of features which competitor products offer, such as:
- Multiple currencies
- On-the-fly product comparisons
- Saved shopping lists
- Extra order status and functionality integration with proprietary call center products
which the ecommerce module does not currently accommodate.
However, none of these products completely encapsulate the functionality of the ecommerce module, and any shortcomings are more than compensated for by the ecommerce module's tight integration into the ACS. Because the ACS offers such powerful access to information that can quickly be used to build an all-round complete profile of the user, it can offer a much more accurate picture of the site's customers.
Another powerful advantage of using the ecommerce module is the simple fact that it is an open source solution. Whilst Akopia's Tallyman is open source too, the functionality they offer is not as complete (it's very pretty though), and at the end of the day it will still just be an ecommerce package rather than an complete collaborative commerce system.
This section describes the way that placing an order and purchasing a gift certificate has been implemented. It also outlines the tasks will be need to be performed for a bare minimum level of maintenance.
Orders
These are the states that an order can go through:
+-------- IN_BASKET <----------------+ (if authorization fails --
| | | it might also temporarily go
| | | into FAILED_AUTHORIZATION
EXPIRED CONFIRMED------------------+ before returning to IN_BASKET)
|
+-----+------------+
| |
AUTHORIZED_MINUS_AVS AUTHORIZED_PLUS_AVS
| |
+----+--------+----+
| |
PARTIALLY_FULFILLED--->FULFILLED
|
RETURNED
An order can also be put into the VOID state at any time by the site
administrator. (Note: these states are actually stored in all lowercase
in the database, but it's clearer to use uppercase letters in the
documentation).
An order is IN_BASKET when the customer has put items into their shopping
cart on the site but has not indicated an intent to buy (if they stay
there too long they go into the EXPIRED state; "too long" is defined in
the parameters/yourservername.ini file; default is 30 days). When the customer
submits their order, the state becomes CONFIRMED. Only then do we try
to authorize their credit card. If the authorization succeeds, the order
state will be updated to AUTHORIZED_PLUS_AVS or AUTHORIZED_MINUS_AVS.
AVS stands for Address Verification System (for more information on AVS,
read the ecommerce
chapter of Philip & Alex's Guide to Web Publishing). Because AVS
is flaky and unreliable, we treat AUTHORIZED_PLUS_AVS orders the same as
AUTHORIZED_MINUS_AVS orders. If an order fails, it goes back into the
IN_BASKET state and the customer is given another chance to enter their
credit card information.
Problems occur if we don't hear back from CyberCash or if they give us a result that is inconclusive. In cases like this, the order state remains in the CONFIRMED state. A scheduled procedure sweeps the database every once in a while looking for CONFIRMED orders that are over 15 minutes old and tries to authorize them. If the authorization succeeds, the order is put into the AUTHORIZED_PLUS_AVS or
AUTHORIZED_MINUS_AVS state. If it fails, it is temporarily put into
the FAILED_AUTHORIZATION state so that it can be taken care of by
a scheduled procedure which sends out email to the customer saying
that we couldn't authorize their order and then saves the order for them (a saved order is one in the IN_BASKET state with saved_p='t'; it can be retrieved easily by the customer later).
Once an order is authorized it is ready to be shipped. An order
which has some of its items shipped is PARTIALLY_FULFILLED and an order which has all of its items shipped is FULFILLED. It remains in the FULFILLED state unless all of the items in the order are returned,
at which time it becomes financially uninteresting and goes into the
RETURNED state.
Individual items in an order also go through a series of states:
IN_BASKET -----------+
| |
TO_BE_SHIPPED EXPIRED
|
SHIPPED
|
ARRIVED
|
RECEIVED_BACK
An item starts out in the IN_BASKET state. When the order it's in becomes authorized, the item becomes TO_BE_SHIPPED. Because partial shipments can be made on orders, SHIPPED is a state of the individual items, not of the order. There is currently no mechanism for
putting an item into the ARRIVED state but it could be used if you were to set up a
method of data exchange with FedEx's database to get the actual arrival date and arrival
detail for each of the packages you ship. If the customer returns an item to you, it is
put into the RECEIVED_BACK state. Like orders, individual items can also be put into the VOID state (e.g. if the customer changes their mind or if you run out of stock before you can ship it).
What can be done with orders?
On an individual order:
- Add comments to it
- Record a shipment
- Process a refund
- Add items (provided it hasn't shipped yet and it was paid for using a credit card instead of using a gift certificate)
- Change the shipping address
- Change the credit card information
- Void it
Gift Certificates
Whether or not customers are allowed to order gift certificates is configurable in the parameters .ini file.
These are the states that a purchased gift certificate goes through:
CONFIRMED
|
+-------------------+---------------------+
| | |
AUTHORIZED_PLUS_AVS AUTHORIZED_MINUS_AVS FAILED_AUTHORIZATION
Regardless of whether you allow customers to purchase gift certificates, you
can always issue gift certificates to your customers. Gift certificates
that you issue automatically go into the AUTHORIZED state.
There are a few fundamental ways in which purchased gift certificates
differ from assigned gift certificates. Purchased gift certificates are
sent to some recipient who may or may not be a registered user of the
system, along with a claim check. These gift certificates must be claimed
upon order checkout in order to be used. Issued gift certificates, on the
other hand, are given to registered users of the system and are put
directly into their gift certificate balance. There is no need for them
to be claimed because there is no ambiguity about who the gift certificates
belong to.
All gift certificates have an expiration date (this is necessary so that your
liability has a definite ending point). A customer's gift certificate
balance is equal to the sum of all the unused portions of each non-expired
gift certificate they own. When their gift certificate balance is applied
toward a new order, the gift certificates that expire soonest are the first
to be applied.
What can be done with gift certificates?
Site Upkeep
Besides the most important task of filling orders, there are some other things that need to be done occasionally.
Dealing with Problems
A log of potential problems is maintained by the system when it comes across issues that it is unable to resolve. These (typically infrequent) problems will need to be resolved by hand.
VI. The Data Model
Big, Friendly Giant
The ecommerce data model is pretty big, but it isn't scary. The majority of the tables are audit tables for tracking activity when data is entered or modified. Everything is well commented, so reading through it should not be too painful at all.
The interesting tables that you might want to look at are:
ec_products. This holds all the information about an individual product, including its price, search keywords, associated shipping costs, weight... pretty much everything you could possibly want to know about a product.
ec_product_purchase_comb tracks the products most commonly purchased at the same time as a particular product_id.
ec_product_series_map stores which products are actually a series of other products rather than a product themselves (e.g., a collection of books).
ec_addresses holds the address and contact information for customers.
ec_creditcards holds credit card information. Unless the administrator configures things differently, credit card information is only stored until an order is completely fulfilled (and this is the recommended approach). This has to be done because a new charge may need to be authorized if only a partial shipment is made (CyberCash's API only allows us to either capture the amount that a charge was authorized for or to capture nothing at all - we can't capture any amount in between; therefore, we are forced to do a new authorization for each amount we are going to charge the user).
ec_user_classes stores all the groups of user that have been defined, such as students, corporations, etc.
ec_product_user_class_prices will hold any special offer prices that are being offered to a user class.
ec_product_recommendations stores recommended products, where this recommendation should be displayed and whether the recommendation is active.
ec_user_class_user_map tracks which users are members of which classes. It is perfectly fine for a user to be a member of more than one class.
ec_product_links stores uni-directional links between two products from one to the other, for the purposes of cross-selling. If a link is to be bi-directional, two entries will be made in the table, one for each direction.
ec_product_comments contains product comments submitted by users; if a user's comment has been approved for appearing on the site (although the administrator can disable the need for approval) then approved_p will be t.
ec_product_reviews contains professional reviews about the products.
ec_category_product_map keeps track of which products belong in which category. Note that it is fine for a product to belong to more than one category.
- Similarly,
ec_subcategory_product_map and ec_subsubcategory_product_map keep track of products in subcategories and subsubcategories.
ec_category_template_map records the template that a category has associated with it. This is a many-one relationship, i.e. one template can be associated with many categories, but a category can only have one template at most.
ec_custom_product_fields keeps track of the extra custom fields that have been added to the standard product attributes for any one product type.
ec_orders is an important one; guess why :-) Seriously, all the columns in this table are pretty self-evident. Nevertheless it is worth mentioning.
ec_shipments records every shipment that is made; note that several shipments may be made for one order in ec_order.
ec_items stores all items that make up an order.
ec_admin_settings holds some settings that the administrator can configure; most notably, way that shipping is calculated. Note: most of the configuration is done in the /web/yourserver/parameters/yourserver.ini
ec_gift_certificates holds all the gift certificates that are either bought for or issued to someone.
ec_gift_certificate_usage stores how much of each gift certificate has been used up.
Hooking up with the Customer Service Model
Note that the customer service module is designed to be used in conjunction with the ecommerce module (something that we definitely recommend, as you will want to track your users problems efficiently, or else they will leave you). The columns order_id and gift_certificate_id in the ec_customer_service_issues table should reference the ec_orders and ec_gift_certificates table respectively.
VII. Transactions & Implementation Details
There are many legal transactions that take place in this module; enough for a book in itself. Perhaps a more useful thing to know how a few things are handled under the hood. This section covers the implemented details for:
- Financial transactions
- Gift certificates
- Order states
- Shopping carts
- Credit card pre-checking
- Automatic emails
- Storage of credit card numbers
- Price calculation
- Financial Transactions
A financial transaction is inserted whenever a credit card authorization to charge or refund is made. These transactions may or may not be carried through to fulfillment. The specifics:
When an order is placed, an authorization is done for the full cost of the order, so a row is inserted into ec_financial_transactions.
This row has a unique transaction_id and it is tied to the order using the order_id. The charge isn't captured yet (it isn't until the items ship).
When a shipment is made, if it's a full shipment of the order, the financial transaction inserted when the order is first placed is ready to be captured (to_be_captured_p becomes 't' and the system attempts to mark and capture it).
However, if only a partial shipment is made, a new authorization has to be made (therefore a new row is inserted into ec_financial_transactions, to_be_captured_p is set to 't' and the system attempts to mark and capture it).
When a refund is made, a row is also inserted into ec_financial_transactions.
A refund is only inserted if it is definite that it needs to be captured, so there is no need to set to_be_captured_p if transaction_type='refund'.
Scheduled procs go around and do the follow-through (making sure everything is marked/settled) for every transaction that needs to be captured.
- Gift Certificates
Each customer has a gift certificate balance (it may be $0.00), which you can determine by calling the PL/SQL function ec_gift_certificate_balance. Different chunks of a customer's balance may expire at different times because every gift certificate that is issued has an expiration date.
When the system applies a customer's gift certificate balance to an order, it begins by using the ones that are going to expire the soonest and continues chronologically until either the order is completely paid for or until the customer's gift certificates run out. If only part of a gift certificate is used, the remaining amount can be used later.
If a customer purchases a gift certificate for someone else, the recipient
(who may or may not be a registered user of the site) is emailed a claim
check that they can use to retrieve the gift certificate and have it
placed in their gift certificate balance. Note: "retrieving" a gift
certificate is equivalent to inserting the user_id of the
owner into ec_gift_certificates. Retrieved gift certificates
always belong to registered users because gift certificates can
only be retrieved during the course of placing an order, at which time
an unregistered user becomes registered.
Site administrators can issue gift certificates to customers at will.
In this case, no claim check is generated. The gift certificate is
automatically assigned to that user_id.
- Order States
Order states are discussed in detail in Operation of the Ecommerce Module. That should be read to understand the
concepts of order states and item states and to see the finite state machines
involved.
Below is a diagram of what order state the order should be in
given the item state of the items in that order. This diagram only
covers the order states VOID, PARTIALLY_FULFILLED, FULFILLED, and
RETURNED. All other order states are grouped under OTHER. In all other
order states, the items are of a uniform item state,
so it is either quite obvious what the order state will be or it is completely
independent of what the order state will be.
An "X" in a column implies that there is at least one (possibly many) item
in that item state.
| Item State | Order State |
|
VOID
|
RECEIVED_BACK
|
SHIPPED
|
OTHER
|
|
X
|
X
|
X
|
X
|
PARTIALLY_FULFILLED
|
|
X
|
X
|
X
|
0
|
FULFILLED
|
|
X
|
X
|
0
|
X
|
PARTIALLY_FULFILLED
|
|
X
|
X
|
0
|
0
|
RETURNED
|
|
X
|
0
|
X
|
X
|
PARTIALLY_FULFILLED
|
|
X
|
0
|
0
|
X
|
OTHER
|
|
X
|
0
|
0
|
0
|
VOID
|
|
0
|
X
|
X
|
X
|
PARTIALLY_FULFILLED
|
|
0
|
X
|
X
|
0
|
FULFILLED
|
|
0
|
X
|
0
|
X
|
PARTIALLY_FULFILLED
|
|
0
|
X
|
0
|
0
|
RETURNED
|
|
0
|
0
|
X
|
X
|
PARTIALLY_FULFILLED
|
|
0
|
0
|
0
|
X
|
OTHER
|
- Shopping Cart Definitions
- Shopping Cart
- An IN_BASKET order with the same
user_session_id as in the user's cookie.
- Saved Cart
- An IN_BASKET order with the
user_id filled in, no user_session_id filled in, and saved_p='t'
- Abandoned Cart
- An IN_BASKET order with saved_p='f' and a user_session_id that doesn't correspond to the user_session_id in anyone's cookie (e.g. the user's cookie expired or they turned cookies off). There's no way of determining whether a shopping cart has been abandoned. These are different from expired orders (see below).
- Expired Cart
- Carts are automatically put into the order state EXPIRED if they're still IN_BASKET after N days, where N is set in the .ini file)
- Credit Card Pre-Checking
Before credit card information is sent out to CyberCash for authorization,
some checking is done by the module to make sure that the credit card
number is well-formed (using the procedure ec_creditcard_precheck which can be found in /tcl/ecommerce-credit). The procedure checks the length of the credit card number, makes sure
it starts with the right digit for the card type, and does a LUHN-10
check (that's a checksum which can't determine whether the number is a
valid credit card number but which determines whether it's even possible
for it to be a valid credit card number).
This procedure only encompasses the three most common credit card types:
MasterCard, Visa, and American Express. It can quite easily be extended
to include other credit card types.
- Automatic Emails
When you install the system, there are 7 automatic emails included that
are sent to customers in common situations (e.g., "Thank you for your
order" or "Your order has shipped"). If a site administrator adds a
new email template using the admin pages, you will have to create a
new procedure that does all the variable substitution, the actual
sending out of the email, etc. This should be easy. Just copy any
one of the 7 auto-email procedures in /tcl/ecommerce-email (except
for ec_email_gift_certificate_recipient, which is unusual).
Then invoke your new procedure anywhere appropriate (e.g. the email that
says "Thank you for your order" is invoked by calling
ec_email_new_order $order_id after the order has been
successfully authorized).
- Storage of Credit Card Numbers
Credit card numbers are stored until an order is completely fulfilled.
This is done because a new charge might need to be authorized if a
partial shipment is made (CyberCash's API only allows us to either capture the amount that
a charge was authorized for or to capture nothing at all - we can't capture any
amount in between; therefore, we are forced to do a new authorization
for each amount we are going to charge the user). A new charge also might
need to be authorized if a user has asked the site administrator to add
an item to their order.
If you've decided not to allow customers to reuse their credit cards,
their credit card data is removed periodically (a few times a day) by
ec_remove_creditcard_data in
/tcl/ecommerce-scheduled-procs (it removes credit card numbers for
orders that are FULFILLED, RETURNED, VOID, or EXPIRED).
If you've decided to allow customers to reuse their credit cards, their
credit card information is stored indefinitely. This is not recommended
unless you have top-notch, full-time, security-minded system administrators.
The credit card numbers are not encrypted in the database because there
isn't much point in doing so; our software would have to decrypt the
numbers anyway in order to pass them off to CyberCash, so it would be
completely trivial for anyone who breaks into the machine to grep for
the little bit of code that decrypts them. The ideal thing would be
if CyberCash were willing to develop a system that uses PGP so that we
could encrypt credit card numbers immediately, store them, and send them
to CyberCash at will. Philip and Alex's Guide to Web Publishing says:
What would plug this last hole is for CyberCash to give us a public key. We'd encrypt the consumer's card
number immediately upon receipt and stuff it into our Oracle database. Then if we needed to retry an
authorization, we'd simply send CyberCash a message with the encrypted card number. They would decrypt the
card number with their private key and process the transaction normally. If a cracker broke into our server, the
handful of credit card numbers in our database would be unreadable without CyberCash's private key. The same
sort of architecture would let us do reorders or returns six months after an order.
CyberCash has "no plans" to do anything like this.
Note 1: If you or the company you work for are very powerful or
influential, perhaps you can put a little fire under CyberCash's bum
to get them to make that change (Levi's couldn't convince CyberCash
when we were doing a project for them). It wouldn't be that hard for CyberCash to implement it.
Note 2: The above discussion does not mean that the credit
card numbers go over the network unencrypted. CyberCash's closed-source
software on your machine encrypts the numbers immediately before sending them out.
Note 3: If you want to let your customers reuse their old credit cards,
you can reduce some of the risk by manually removing old credit card data
once in a while (at least then there will be fewer numbers in your database for
the crackers to steal). To clear out the unnecessary credit card data, just
run a procedure like ec_remove_creditcard_data (in
/tcl/ecommerce-scheduled-procs) but get rid of the if statement that
checks whether SaveCreditCardDataP is 0 or 1.
Note 4: You may wish to encrypt credit card numbers in Oracle so that any data exports you do don't contain plaintext numbers. This provides little extra security since, as discussed above, they key will need to be stored on the server, but at least your exports will be safer. The encryption can be implemented with minimal effort by using Oracle's dbms_obfuscate package.
- Price Calculation
The site administrator can give the same product different prices for
different classes of users. They can also put products on sale over
arbitrary periods of time (sale prices may be available to all
customers or only to ones who have the appropriate offer_code
in their URL).
The procedure ec_lowest_price_and_price_name_for_an_item in /tcl/ecommerce-money-computations determines the lowest price that a given user
is entitled to receive based on what user classes they're in and what
offer_codes they came to product with. Their offer_codes are stored, along with their user_session_id, in
ec_user_session_offer_codes (we decided to store this in
the database instead of in cookies because it was a slightly more efficient
method, although either implementation would have worked). One minor
complication to this is that if a user saves their shopping cart, we want
them to get their special offer price, even though they may be coming
back with a different user_session_id; therefore, upon retrieving saved
carts, the offer_codes are inserted again into
ec_user_session_offer_codes with the user's current
user_session_id (we had to associate offer_codes
with user_session_id as opposed to user_id
because someone with an offer_code shouldn't be prevented
from seeing the special offer price if they haven't logged in yet).
VIII. API
All the ecommerce procedures are defined in /tcl/ecommerce-*.tcl and /www/arsdigita/doc/sql/ecommerce-plsql.sql, and are pretty well documented. Here are a few useful ones:
| Function |
Name and File |
| Returns the remaining balance of a gift certificate. |
ec_gift_certificate_balance in ecommerce-plsql.sql |
| Checks that a credit card number is well-formed. |
ec_creditcard_precheck in /tcl/ecommerce-credit.tcl |
| Removes credit card numbers for orders that are FULFILLED, RETURNED, VOID, or EXPIRED.
| ec_remove_creditcard_data in /tcl/ecommerce-scheduled-procs.tcl |
| Determines the lowest price that a given user is entitled to receive based on what user classes they're in and what offer_codes they came to product with |
ec_lowest_price_and_price_name_for_an_item in /tcl/ecommerce-money-computations.tcl |
| Returns the total price for an order, given an order_id without shipping, tax, or gift certificates |
ec_total_price in ecommerce-plsql.sql |
| Returns the status of an order for a customer |
ec_order_status in /tcl/ecommerce-utilities.tcl |
IX. User Interface
Considerations for the user
The most important issues that played a part in deciding how the ecommerce module's interface should be structured were accessibility and consistency.
Accessibility means, in more specific terms:
- Access to functionality
Two good examples of this are the site-wide product search and the shopping cart. The decision was made to put these two features at the head and foot of all user pages. Why? Because if a user has to take time and effort to find the functionality they want, they will just go somewhere else where it will cost them less effort. To that end, easy access to the functions they want is of primary importance.
- Access to information
All the user pages also provide a link to the customer's account page. From there they can see details of their complete order history. They can also see details of orders that they have outstanding.
The addition of the product recommendations feature also improves the quality of the user interface. Because products can be split into categories, there is a danger that users may pass by a category completely and never become aware of the items that are for sale. Per-category product recommendations provide a way around this by 'pushing' good products to the top of the pile for the user to notice.
Considerations for the site-wide administrator
The admin index page offers links to all the areas of administration in the ecommerce module, but more importantly displays one key statistic relevant to that area, so that administrators can instantly see if there is a particular area that requires attention, e.g. there is 1 unresolved problem with the site. This extra information makes it much easier to check over the status of the site `at a glance', whilst not interfering with the purpose of the index page.
Areas that might be handled by different people have been kept together, to save administrators having to `jump about' from one set of links off the index page to another. All order information, including reports, is found under the `Orders / Shipments / Refunds' section.
The user interface was otherwise designed based on what had been successful on other ecommerce web sites, and the specifications of clients who we had in mind when designing the ecommerce module.
X. Configuration Parameters
The following items need to be installed / configured / executed before the ecommerce module will work properly:
- This module requires Oracle 8i.
- Install CyberCash by following the instructions for the ArsDigita Shoppe.
(Note: it can take a few weeks for your bank and CyberCash to get your account ready, so get started on this right away)
- Ensure that these files are present in the ACS:
- Data model in /arsdigita/doc/sql/ecommerce*.sql
- Documentation in /arsdigita/doc/ecommerce*
- Scripts in /ecommerce/
- Admin scripts in /admin/ecommerce/
- ADP templates in /templates/ecommerce/
- Tcl procs in /tcl/ecommerce-*
- Ecommerce section of .ini file in parameters/
- Ecommerce section of /admin/users/one
- Ensure that the tools package, the style package, and the audit tool are installed.
- Load the data model into Oracle.
- ImageMagick must be installed (either in /usr/local/bin/ or elsewhere if you're running your server chrooted)
if you want thumbnails to be automatically created. For information on ImageMagick (which is free), see Chapter 6: Adding Images to Your Site of Philip and Alex's Guide to Web Publishing.
- Make the directory
/web/yourserver/data/ecommerce/product to hold the products' supporting files (it's outside the web root so that no uploaded supporting files can be executed). The directory has to be writable by nsadmin. (You can change the directory by editing EcommerceDataDirectory and ProductDataDirectory in your parameters .ini file.)
- Based on the site owner's publishing decisions, the ecommerce parameters
in /web/yourserver/parameters/yourserver.ini need to be edited. The following is a description of what each of the parameters means. If you don't understand some of these descriptions, read on. They should make sense after you've finished reading this section.
| Parameter | Setting in .ini file at [ns/server/atst_sarah/acs/ecommerce] |
|---|
| The unit of currency, e.g. USD, to use throughout the site |
Currency |
| The unit of weight, e.g. lbs, to use throughout the site |
WeightUnits |
| How many products to display per page when customer is browsing (default 10) |
ProductsToDisplayPerPage |
| Whether to allow users to write public comments about products and, if so, whether the comments need to be approved by admins before they appear on the site |
ProductCommentsAllowP |
| Whether product relationships (e.g. "people who bought product A also bought products B, C, and D") should be calculated and displayed |
CalcProductRelationshipsP |
| Whether users can see what classes they belong to |
UserClassUserViewP |
| Can users request to be placed in classes via the web site? |
UserClassAllowSelfPlacement |
| Whether users automatically become members of classes they have requested to join (if not, they have to be approved by admins) |
UserClassApproveP |
| What percentage of the shipping charges should be refunded by default if a customer returns their purchases |
ShippingRefundPercent |
| Whether express shipping is available |
ExpressShippingP |
| Whether to save credit card data (if so, customers can reuse their credit card with one click; if not, the credit card number is deleted after the order has shipped) |
SaveCreditCardDataP |
| How large you want the automatically-generated thumbnail images of the products to be (you can specify either the width or the height, in pixels; the dimension you don't specify will vary based on the original image's size) |
ThumbnailWidth, ThumbnailHeight |
| What product stock messages you want to be able to choose from when adding/editing products (e.g. "Out of Stock", "Usually Ships Within 24 Hours", "Usually Ships Within 2-3 Days", etc.) |
Examples:
StockMessageO=Out of Stock
StockMessageQ=Usually Ships Within 24 Hours
StockMessageM=Usually Ships Within 2-3 Days
StockMessageS=Usually Ships Within 2-4 Weeks
StockMessageI=In Stock |
| The number of days a user's shopping cart will stay valid before it goes into the `expired' state |
CartDuration |
| Whether to allow pre-orders for items that are not yet available |
AllowPreOrdersP |
| The email address that will be used for all email sent from the system to the customers |
CustomerServiceEmailAddress |
| Whether people fulfilling the orders should be alerted if there's a problem re-authorizing a credit card for payment (which happens when orders are shipped) -- you'll want them to be alerted if they're in a position to do anything about the problem (e.g. abort the shipment); otherwise, there's no need to alert them because the problem will still be logged so that someone else can take care of it |
DisplayTransactionMessagesDuringFulfillmentP |
| Whether customers are allowed to purchase gift certificates for others |
SellGiftCertificatesP |
| The minimum value a gift certificate is allowed to be |
MinGiftCertificateAmount |
| The maximum value a gift certificate is allowed to be |
MaxGiftCertificateAmount |
| The number of months a gift certificate will last before it expires |
GiftCertificateMonths |
- The page templates display correctly only if you're using fancy ADP parsing.
To do this, make sure that this is in your server .ini file (replacing yourserver
with the name of your server):
[ns/server/acs/adp/parsers]
fancy=.adp
- Qmail must be installed on your system (it may require special setup
if you're running your server chrooted).
To get the ecommerce module up and running, you will need to perform the following tasks. See the requirements document user scenarios for how this is done.
- Set up product categorization (/admin/ecommerce/cat/)
- Set up your shipping cost rules (/admin/ecommerce/shipping-costs/)
- Set up your sales tax rules (/admin/ecommerce/sales-tax/)
- If desired, add custom product fields at /admin/ecommerce/products/custom-fields.
- Create new product display templates at (/admin/ecommerce/templates/)
- Set up user classes (/admin/ecommerce/user-classes/)
- Enter your products into the database either using the form at /admin/ecommerce/products/add or the bulk uploader utilities at /admin/ecommerce/products/upload-utilities.
- After you've added a product, there are a variety of things you can
do to it, such as:
- Add any number of professional reviews.
- Add "cross-selling links" so that the customer always sees a link to another given product when they're viewing this product, or vice versa.
- Put the product on sale or create "special offers".
- Add product recommendations (/admin/ecommerce/products/recommendations).
- Modify the email templates (/admin/ecommerce/email-templates/), which are used when the system sends out automatic email
to customers.
- The layout for all the pages in the site is created using templates
which are stored in the directory /web/yourservername/templates/ecommerce/
(with the exception of product which, as discussed above, uses a different
templating system to allow for different templates for different products).
If you are unhappy with the look of any of the pages in your site, there's
a good chance that it can be changed simply by editing the corresponding
template.
XI. Acceptance Tests
After you have completed the setup tasks in
Technical Details of the Ecommerce Module, here are some good tests of the things most likely to be broken:
- Go to http://yourserver/ecommerce/ . If the page is broken, it's because the data model wasn't automatically loaded with the rest of the ACS (the ecommerce data model is omitted intentionally to save publishers who aren't doing ecommerce from having a gigantic number of unused tables). Solution: load the two files ecommerce.sql and ecommerce-plsql.sql with
sqlplus username/password < ecommerce.sql (same for ecommerce-plsql.sql). If you get an error while loading the data model, it may mean that you're not using Oracle 8i (required).
- Go to https://yourservername/admin/ecommerce/products/add (or http:// instead of https:// if you don't have SSL) and try to add a product without a picture. If you get an error on add-2, it's probably because the directory to hold auxiliary product info cannot be created.
Make sure the directory (specified by the parameters EcommerceDataDirectory and ProductDataDirectory) exists. By default, this is /web/yourserver/data/ecommerce/product/ . Also make sure the permissions are correctly set (AOLserver has to be able to create new directories inside of this directory).
- Now try to add a product with a picture. If it can't be created, that may mean that ImageMagick is not installed or is not at the expected path (/usr/local/bin/convert).
- Go to http://yourserver/ecommerce/ and click on one of the products you just added. Add it to your shopping cart and try to check out (using a real credit card). If all has gone well, you will never see finalize-order (you'll be redirected immediately to thank-you), but if you get an error on finalize-order that may mean that CyberCash has not been correctly set up.
Once all this is working, you can further test your system:
- Follow the setup instructions as described in the use cases in the Ecommerce Module requirements document
- Go to the user pages at http://yourserver/ecommerce/ and place
orders, write reviews of products, join mailing lists, view your
account, etc.
- Return to the admin pages and do some maintenance (take a look at "Site upkeep" in section V).
XII. Future Improvements/Areas of Likely Change
The ecommerce module is intended to have the all following features integrated into it:
- Support for multiple retailers. (Includes an extranet for approved retailers
to upload price and stock information about products they have for sale.)
- Integration with the ACS Graphing Package to show colorful sales reports.
XIII. Authors
The ecommerce module was designed and implemented by Eve Andersson. Eve is the owner of the ecommerce module for all pre-4.0 versions of the ACS; when it moves to version 4.0 Eric Lorenzo will be taking ownership of the project.
The original documentation was also written by Eve; it has been updated, reformatted and augmented by Sarah Ewen.
XIV. Revision History
acs-docs@arsdigita.com
