Django 1.2 E-commerce

An emerging trend in recent years has been the arrival of powerful new web development frameworks, of which Django is a premier example. These frameworks are designed to build content-rich websites quickly and easily. They have different individual philosophies, but all seek to solve one main problem—simplifying the repetitive and time-consuming development tasks that all websites and applications demand.

Solving this problem achieves two things:

Django is specifically designed to optimize for these characteristics. One unofficial motto is "don't repeat yourself". This means, do not reproduce the wheel, instead focus on the unique aspects of your application that provide new and profitable functionality.

Many of the problems that affect content-driven websites actually affect all businesses, even traditional brick-and-mortar ones. For instance, consider a restaurant that offers a menu of daily specials or whose menu changes frequently based on the selection of fresh food available in the city or region. To market this menu on the Web requires tools for quickly producing and updating content. We can imagine similar examples in the retail world, where product inventories ebb and flow depending on seasons, styles, and trends.

In the online world, this kind of content production is often taken for granted. The entire business of Amazon.com is built around a large, structured, and highly-detailed database of product information, along with customer ratings and reviews. Netflix's enormous selection of movie rentals would not be possible without an equally large content database.

Django provides an excellent basis to build e-commerce websites in part because of its ability to handle these content problems. Content is one of Django's specialties, having been created to serve the needs of the newspaper industry. It provides many out-of-the-box tools to handle these demands. For example, the built-in admin interface (as shown in the following screenshot) can automatically manage any application's data, in a way that anyone with a web browser can access. The admin tool can be attached to any Django project with a trivial amount of effort and just a few lines of code.

Handling content so effectively is possible because of Django's excellent Object-relational Mapper (ORM). Data-driven approaches to web development have been around for a decade. Originally this involved lots of tedious SQL code, but over a period of time much of this has been abstracted away in libraries or framework code. Django's ORM is arguably one of the best abstractions created thus far and its query interface is often cited as a killer feature. It's even possible to use the ORM portion of Django in standalone mode and take advantage of its expressive power. Django's ORM has even inspired other projects, most notably Google's App Engine project, which borrows features and syntax. We will highlight many features of Django's ORM throughout this book.

There are many other advantages to Django as an e-commerce platform. Rapid development tools, in general, mean cost reductions for initial development and long-term maintenance. Decreasing these costs is another specialty of Django. As an example, in Chapter 2, Setting Up Shop in 30 Minutes, we will create a fully functional web-based store, from scratch, in 30 minutes.

What about competing frameworks? What advantages does Django have over them? A big one is Python. Without stoking religious debate, Python, by its own merits, is an excellent programming language for the Web. Django is built in Python and features many "Pythonic" tactics at the core of its design. The language includes numerous modules that are useful to web developers, including a URL manipulation library, XML parsing tools, and web service tools. The Python community is massive, and provides additional modules and tools. Python developers can frequently avoid reinventing the wheel because either the language or the community already provides what they need.

Django is flexible and able to handle content as well as other e-commerce problems, such as shopping carts, user management, and payment processing. Built-in support for cookie-based user sessions and basic account authentication gives Django developers a major head-start over developers who must implement these from scratch. Submitting orders and processing payments through any of the Web's payment services is also very easy, as we will see in Chapter 4, Building Payment Processors.

All of these factors exhibit a common theme: competitive advantage. Cost savings, code reusability, and flexibility of use all allow businesses to do more with less. E-commerce sites built using Django can be deployed faster and respond quickly to change.

There are a few important criticisms of Django that might affect the decision to use it in your specific projects. First, it's relatively new and has yet to enter the mainstream software community. It's also open source, which unfortunately still prevents its use in some corporate environments, due to concerns over support, maintenance, and security. And finally, it is written in a different language and paradigm than many current mainstream e-commerce products. There are armies of Java developers because many e-commerce applications are built with it. By comparison there are fewer skilled Python developers.

The term "e-commerce" encompasses a wide array of applications. It includes everything from simple shopping-cart powered Internet stores to large-scale business-to-business systems. Fortunately, many of these applications begin with a similar set of technical needs. These include storing, updating, and retrieving important information from a data store (usually a database), rendering information in a common format such as HTML or XML, and interacting with users who will consume, manipulate, and process the information.

These basic needs happen to align with a design pattern known as Model-View-Controller (MVC). MVC is intended to simplify the construction of applications by dividing it up into three manageable parts: the model, which is focused on storing and structuring data, the view, which presents data exposed by the model, and finally the controller, which provides a means of interacting with and manipulating model data.

Django follows this design pattern, but prefers to call it Model-Template-View (MTV). It retains the model concept from MVC, but refers to the controller portion as "views" while replacing the MVC view with what it calls the "template". These can be standard HTML templates with added functionality provided by a Django specific template language. This language can be replaced with other template languages, such as Python's popular Jinja or Cheetah template systems. Django can also use templates to render other formats such as XML or JSON.

Django's MTV pattern is a powerful approach, especially for designing web-based applications. It allows components to be created in a reusable and modular way so that pieces of the system can be upgraded or replaced without disrupting the others. This simplifies development. By focusing on the abstraction each MTV component represents, developers need not worry about the implementation details of the other components, only how to interact with them. Django's implementation of the MTV pattern should be a comfort to e-commerce developers because it is a well known and solid foundation to build upon.

Another critical e-commerce component is the payment processor, sometimes called a payment gateway or a checkout system. There are hundreds of companies offering payment solutions today. Almost all process credit cards, but some allow for "e-checks" and other more obscure payment types. These payment processors generally compete on the fees they charge per transaction. The standard rates usually involve a small fee per transaction in addition to a percentage of the transaction (usually between 1.5 percent and percent).

PayPal has historically been one of the major web-based payment processing companies and continues to process a large majority of Internet payments. There are many alternatives, however, and two of the most formidable are powered by Google and Amazon. In my opinion the latter two processors are more developer-friendly and highly recommended. Google's processor is called "Checkout" and Amazon has built a payment processor called "Flexible Payment Services" as part of their suite of web-services tools. We will build Django apps for these payment processors in Chapter 4.

The traditional means of bundling products for sale and submitting them to a payment processor involves a shopping cart. Usually, this involves the customer who selects products from the catalog web pages by adding them to their shopping cart, and then checks out with their payment method via a payment processing service. We will begin implementing a Django-based shopping cart in Chapter 3, Handling Customers and Their Orders.

Django is able to interface with these payment processing systems easily, thanks in part to the availability of tools and modules in the Python community that simplify common tasks. Python code examples and in some cases entire wrapper modules are available for many payment processing solutions. In addition, Django's template system can be used to simplify rendering of XML and other data files required to submit payment requests.

Shopping carts are also relatively simple to implement using Django. The built-in session's framework provides an easy way to store cart and checkout information between pages in a user's session. We will build our sample project on this approach later in this book.

Django takes a disciplined approach to constructing websites. It has many conventions and design constructs that sometimes are not obvious to new developers. All frameworks employ their own philosophy in solving problems and the philosophical underpinnings tend to appear throughout. Some developers feel restricted by this effect, but it's a natural trade-off in the usage of frameworks (or any code library). If you leverage a framework in your development, it will steer you in certain directions. Much like a programming language can inform one's view of the world, so can frameworks.

There's an old proverb: When in Rome, do as the Romans do. The same can be said for development platforms. Some developers tend to apply their own philosophies in the context of a library or framework and end up frustrated. There is a great deal of grey-area here, of course, but it is important to spend some time understanding the Django-way of application development.

Much of Django's approach is informed by Python itself. Python is a programming language built on distinct philosophies. Its creator, Guido van Rossum, baked in many of these design decisions from the beginning.

Django's introductory tutorials take a simplified approach to getting up and running on the platform. They introduce the concept of a Django project using django-admin.py and the startproject command. This is the easiest way to get going, but it hides a significant amount of activity that goes on under-the-hood. For the project that we will build throughout this book, we will take a more sophisticated approach, by thinking of our Django projects as a collection of normal, plain Python modules. Django calls these modules "apps."

In some ways the term "app" is unfortunate because it hints at something large, a full-blown application. But in Django, apps are usually small and simple. Properly designed apps can be plugged together to form powerful combinations, with each app solving its own portion of a larger problem.

Apps, like any good Python module, encapsulate a specific set of functionality; this is usually by focusing on a small problem and solving it. This is actually a well-established pattern of software design, originating in the UNIX operating system. Past and current versions of UNIX provided numerous tiny programs that solved a simple problem: pattern matching with grep, word counting with wc, and so on.

By emphasizing this approach in our projects, we can take full advantage of Django's rapid development philosophy. It will lead to better code and cleaner designs.

Let's consider the role of Django apps in our e-commerce platform. The "solving small problems approach" fits well; many pieces of our e-commerce project will be common across multiple sites. By keeping our apps small and focused, we will be able to assemble the individual components in different ways for different projects.

For example, two e-commerce stores may share the same payment processor, but have entirely different needs for interacting with their customers. One site might need the ability to send an e-mail newsletter, while the other would not. If we were to build our Django project in large, monolithic sections, it would require more time and effort to satisfy the needs of these two projects. If, however, we use small, tiny pieces, we can simply plug-in the parts they have in common and upgrade or build separately the pieces that differ.

In larger development shops, this also allows for the internal reuse of apps across departments or functional groups. It can make code sharing and reuse much easier for in-house teams.

Keeping a modular design has other advantages. When a project decides to change payment processors, the upgrade is much simpler when the processing code lives alone in its own module. We can standardize the interface across all payment processors so other apps can interact with all of them the same way. In Python this is sometimes called "duck typing" and we will explore it more in Chapter 4.

Django's settings file has an important attribute known as INSTALLED_APPS. This is a Python sequence of module names that will be used for the project. In some ways this is Django's secret weapon. Ideally we can deploy dozens of entirely different sites by doing nothing more than creating a new settings file with an appropriate set of modules in INSTALLED_APPS and pointers to different databases and template locations.

Solving small problems with focused Django apps is the best way to achieve these goals. It is important to remember that we will be writing apps, or better yet, normal Python modules. These will be pieces of something larger, not full-blown applications themselves.

Reusability is software engineering's Holy Grail. Unfortunately, over time it has often proven difficult to attain. It's almost impossible to build for reuse on the first try and other times it's just not practical. But it's still an important goal that often does work and leads to many efficiency gains.

In frameworks like Django that utilize an ORM to interact with and store information in a database, reusability faces additional challenges. The object-oriented programming model that is typically the heart of reusable code does not always translate into a relational database. Django's ORM does its best to accommodate this by offering a limited form of inheritance, for example, but it still has many challenges.

Another tendency is to build Django models that store data for overly specific problems. In Chapter 2, we will begin writing models for a product database. It would be very easy to apply model inheritance in an attempt to solve specific problems. For example, we may be tempted to extend a product model into a subclass specific for food: FoodProduct. We then may try and build a subclass specifically for noodles: NoodleProduct. Using inheritance this way often makes sense in other software projects, but when Django maps these models to the database, it can become a mess of entangled relationships and primary keys.

To avoid these issues with inheritance, some Django developers employ various model hacks. This includes things such as pickling attribute data into a text field or otherwise encoding extra attributes into a single field. These are often clever and sometimes very effective solutions, but they can also lead to bad designs and problems later on.

The best advice seems to be to keep things simple. Limit what your app does and the dependencies it needs to serve its core duty. Don't be afraid of developing a large app library.

As mentioned earlier, Django projects are essentially collections of regular Python modules. These are managed using a settings file. We can have more than one settings file for any project—separate production and a development settings file, for example. Django settings files are also code, which adds even greater flexibility. Using import statements, for example, we can include a standard set of Django settings in every project and simply override the ones we need to change.

When using version control software, it's helpful to keep many different settings files. These files can be organized by developer, by server name, or by code branch. Keeping multiple settings files simplifies deployment and testing. A settings file for each test server or individual developer reduces confusion and prevents breakage as developers create and integrate new apps. However, it retains the convenience and safety of version control. Trying to maintain a large project with multiple developers and a single settings file is a recipe for disaster.

With multiple settings files floating around, how does Django know which one to use? In the command line environment for running interactive Python shells or scripts, this is controlled using an environment variable called DJANGO_SETTINGS_MODULE. Once your settings are in place, you can quickly switch back and forth between them by modifying this environment variable. For example:

# switch to our testing settings and run a Django interactive shell
$ export DJANGO_SETTINGS_MODULE=myproject.settings_testing
$ django-admin.py shell
# switch back to our production settings
$ export DJANGO_SETTINGS_MODULE=myproject.settings_production

A convenient alternative to the preceding manual process is to use the settings flag of django-admin.py, which will adjust the settings module appropriately. Switching to production settings looks like this:

$ django-admin.py–-settings myproject.settings_production

A simple set of shell scripts can automate the use of several settings files. The following shell script wraps Django's django-admin.py and sets a specific settings module. This way we can run our project from the command line with a single simple command, like this.

#!/bin/sh
# myproject_dev - a shell script to run myproject in development mode
export DJANGO_SETTINGS_MODULE=myproject.settings_dev
django-admin.py $@ 

For server deployments, the settings file is specified as part of the server configuration. Sometimes this involves changing the environment variable from an Apache config file (SetEnv DJANGO_SETTINGS_MODULEmyproject.settings). On mod_wsgi setups, it usually means modifying the Web Server Gateway Interface (WSGI) script. We will explore these configuration techniques in Chapter 10, Deployment and Maintenance Strategies.

The other core piece of all Django projects is the root URLs file. Again, this is just a code that defines a set of URL patterns using regular expressions. We can include multiple versions (production vs. development) for our project and use normal Python flow-control to make adjustments such as adding special URL patterns when our project has the DEBUG setting turned on.

For larger Django projects, multiple settings and URL files can quickly get out of hand. As a result it is a smart practice to keep project files, such as settings and root URL configurations, completely separate from app code. Structuring your project and app libraries is often dependent on personal taste or company standards, but keeping a modular and loosely-coupled mindset is always beneficial.

Arguably the most important part of preparing a new Django project is choosing a name. All kidding aside, the name is important and in that it has a few restrictions: it cannot be the same as a Django or Python module and it should conform to usual Python module naming conventions (no spaces, dashes, or other illegal characters). Django projects have a tradition of naming themselves after jazz musicians (Django itself refers to Django Reinhardt, a jazz guitarist). The project we will build in this book will be named Coleman.

To start with, create a new directory and place it on your PYTHONPATH. The easiest way to do this is to make a directory in the usual way and then create a corresponding .pth file to copy to your system's site-packages. For example, if you're working directory is /home/jesse/ecommerce-projects, then create a single line file called ecommerce.pth that looks like this:

/home/jesse/ecommerce-projects

Copy the file to your system-wide site-packages directory. This varies based on your operating system and how you installed Python. See your documentation or the value of sys.path from the Python interpreter.

Later on in this book we will examine some tools and techniques to drastically simplify Django and Python project dependencies and layouts. One of these tools is called virtualenv, which can help us to better manage $PYTHONPATH. For now, though, we'll do everything the old fashioned way.

Next we will create our project directory and subdirectories. These will correspond to Python modules in our code. From your working directory create two folders: one for our app library and one for our project. Call the first one coleman and the second ecommerce_book. We will refer to the coleman module as our app library and to ecommerce_book as our project module. Next we will create our first app module by creating a directory called products inside our app library module. We will begin building the products app in Chapter 2. Your final directory structure will look like this:

./coleman
./coleman/products
./ecommerce_book

One final piece of preparation: we must create a Python __init__.py file in each of these module locations. This will look like the following:

./coleman/__init__.py
./coleman/products/__init__.py
./ecommerce_book/__init__.py

Lastly, we will create our settings file and a root URLs file in our project module: ecommerce_book/settings.py and ecommerce_book/urls.py. It is recommended that you copy these files from the companion source code. We will then refer to ecommerce_book.settings in our DJANGO_SETTINGS_MODULE.

The project settings file in the companion code assumes that you have a working sqlite3 installation as the database backend (sqlite3 is included with Python 2.5 and later). You can change this by editing the settings.py file, which you will need to do in order to complete the full path to your sqlite3 database file. This is another file that fits nicely into the projects directory, and by placing it there you gain the ability to use Python's os module to locate it. This is more desirable than hard coding a path into settings.py. To take this approach with the database file, for example, you could define a setting to represent the project's location on the file system, and then append the database name:

PROJECT_HOME=os.path.dirname(os.path.realpath(__file__))
DATABASE_NAME=os.path.join(PROJECT_HOME, 'mydatabase.db')

Once you've made the appropriate changes, run django-admin.pysyncdb to create your initial database.

These instructions assume that you already have a working installation of Django and have the django-admin.py script installed on your system's path. For more information on configuring and installing Django, please see the documentation at djangoproject.org/docs.

Django 1.2 includes a number of new features and very minimal backwards incompatible changes. As of Django 1.0, the Django development team adopted an API stability policy that alleviates the worry about backwards incompatible changes being introduced into the framework throughout the 1.x series of releases. In other words, with few exceptions, if your application runs on version 1.0, it should run on all 1.x versions.

Major new additions to the Django framework in version 1.2 include multiple-database support, improved if template tag, object-level permissions, e-mail backends, and much, much more.

Per the API stability policy, a minimal number of backwards incompatible changes are included in 1.2. Major changes to existing framework features, like the new if tag, necessitate some incompatible changes. In 1.2 these have been very minor and are well documented in the release notes.

The Django 1.2 release notes also include an overview and brief documentation of the new features and functionality available, and are an excellent starting point for projects that are upgrading.

This chapter has outlined several basic tenets of software development using Django. Planning is one of the most important steps for any project, and understanding Django's particular way of approaching web development helps in the planning phase. The Django way includes:

We also explored the reasons why Django makes a powerful platform for e-commerce applications and explained some of the basic needs these applications tend to have in common. In Chapter 2 we will jump right in to starting our project by building a very basic web-based store in 30 minutes.