Some of the things that we will cover in this two-part article by Carlos de la Guardia, author of Grok 1.0 Web Development, includes:

• Need for testing
• Testing in Grok
• Extending the functional test suite provided by grokproject
• Other kinds of testing
• Debugging tools

Grok offers some tools for testing, and in fact, a project created by grokproject (as the one we have been extending) includes a functional test suite. In this article, we are going to discuss testing a bit and then write some tests for the functionality that our application has so far.

Testing helps us avoid bugs, but it does not eliminate them completely, of course. There are times when we will have to dive into the code to find out what's going wrong. A good set of debugging aids becomes very valuable in this situation. We'll see that there are several ways of debugging a Grok application and also try out a couple of them.

Testing

It's important to understand that testing should not be treated as an afterthought.

As mentioned earlier, agile methodologies place a lot of emphasis on testing. In fact, there's even a methodology called Test Driven Development (TDD), which not only encourages writing tests for our code, but also writing tests before any other line of code.

There are various kinds of testing, but here we'll briefly describe only two:

• Unit testing
• Integration or functional tests

Unit testing

The idea of unit testing is to break a program into its constituent parts and test each one of them in isolation. Every method or function call can be tested separately to make sure that it returns the expected results and handles all of the possible inputs correctly.

An application which has unit tests that cover the majority of its lines of code, allows its developers to constantly run the tests after a change, and makes sure that modifications to the code do not break the existing functionality.

Functional tests

Functional tests are concerned with how the application behaves as a whole. In a web application, this means how it responds to a browser request and whether it returns the expected HTML for a given call.

Ideally, the customer himself has a hand in defining these tests, usually through explicit functionality requirements or acceptance criteria. The more formal the requirements from the customer are, the easier it is to define appropriate functional tests.

Testing in Grok

Grok highly encourages the use of both kinds of tests, and in fact, includes a powerful testing tool that is automatically configured with every project. In the Zope world—from where Grok originated—a lot of value is placed in a kind of tests known as "doctests", so Grok comes with a sample test suite of this kind.

Doctests

A doctest is a test that's written as a text file, with lines of code mixed with explanations of what the code is doing. The code is written in a way that simulates a Python interpreter session. As tests exercise large portions of the code (ideally 100%), they usually offer a good way of finding out of what an application does and how. So, if an application has no written documentation, its tests would be the next obvious way of finding out what it does. Doctests take this idea further by allowing the developer to explain in the text file exactly what each test is doing.

Doctests are especially useful for functional testing, because it makes more sense to document the high-level operations of a program. Unit tests, on the other hand, are expected to evaluate the program bit by bit and it can be cumbersome to write a text explanation for every little piece of code.

A possible drawback of doctests is that they can make the developer think that he needs no other documentation for his project. In almost all of the cases, this is not true. Documenting an application or package makes it immediately more accessible and useful, so it is strongly recommended that doctests should not be used as a replacement for good documentation. We'll show an example of using doctests in the Looking at the test code section of this article.

Default test setup for Grok projects

As mentioned above, Grok projects that are started with the grokproject tool already include a simple functional test suite by default. Let's examine it in detail.

Test configuration

The default test configuration looks for packages or modules that have the word 'tests' in their name and tries to run the tests inside. For functional tests, any files ending with .txt or .rst are considered.

For functional tests that need to simulate a browser, a special configuration is needed to tell Grok which packages to initialize in addition to the Grok infrastructure (usually the ones that are being worked on). The ftesting.zcml file in the package directory has this configuration. This also includes a couple of user definitions that are used by certain tests to examine functionality specific to a certain role, such as manager.

Test files

Besides the already mentioned ftesting.zcml file, in the same directory, there is a tests.py file added by grokproject, which basically loads the ZCML declarations and registers all of the tests in the package.

The actual tests that are included with the default project files are contained in the app.txt file. These are doctests that do a functional test run by loading the entire Grok environment and imitating a browser. We'll take a look at the contents of the file soon, but first let's run the tests.

Running the tests

As part of the project's build process, a script named test is included in the bin directory when you create a new project. This is the test runner and calling it without arguments, finds and executes all of the tests in the packages that are included in the configuration.

We haven't added a single test so far, so if we type bin/test in our project directory, we'll see more or less the same thing that doing that on a new project would show:

$ bin/test
Running tests at level 1 
Running todo.FunctionalLayer tests: 
  Set up 
 in 12.319 seconds. 
  Running: 
...2009-09-30 15:00:47,490 INFO sqlalchemy.engine.base.
Engine.0x...782c PRAGMA table_info("users") 
2009-09-30 15:00:47,490 INFO sqlalchemy.engine.base.Engine.0x...782c () 

  Ran 3 tests with 0 failures and 0 errors in 0.465 seconds. 
Tearing down left over layers: 
  Tear down todo.FunctionalLayer ... not supported 

The only difference between our output to that of a newly created Grok package is in the sqlalchemy lines. Of course, the most important part of the output is the "penultimate" line, which shows the number of tests that were run and whether there were any failures or errors. A failure means that some test didn't pass, which means that the code is not doing what it's supposed to do and needs to be checked. An error signifies that the code crashed unexpectedly at some point, and the test couldn't even be executed, so it's necessary to find the error and correct it before worrying about the tests.

The test runner

The test runner program looks for modules that contain tests. The test can be of three different types: Python tests, simple doctests, and full functionality doctests. To let the test runner know, which test file includes which kind of tests, a comment similar to the following is placed at the top of the file:

Do a Python test on the app.

:unittest:

In this case, the Python unit test layer will be used to run the tests. The other value that we are going to use is "doctest" when we learn how to write doctests.

The test runner then finds all of the test modules and runs them in the corresponding layer. Although unit tests are considered very important in regular development, we may find functional tests more necessary for a Grok web application, as we will usually be testing views and forms, which require the full Zope/Grok stack to be loaded to work. That's the reason why we find only functional doctests in the default setup.

Test layers

A test layer is a specific test setup which is used to differentiate the tests that are executed. By default, there is a test layer for each of the three types of tests handled by the test runner. It's possible to run a test layer without running the others and also to name new test layers to be able to cluster together tests that require a specific setup.

Invoking the test runner

As shown above, running bin/test will start the test runner with the default options. It's also possible to specify a number of options, and the most important ones are summarized below. In the following table, command-line options are shown to the left. Most options can be expressed with a short form (one dash) or a long form (two dashes). Arguments for the option in question are shown in uppercase.

Looking at the test code

Let's take a look at the three default test files of a Grok project, to see what each one does.

ftesting.zcml

As we explained earlier, ftesting.zcml is a configuration file for the test runner. Its main objective is to help us set up the test instance with users, so that we can test different roles according to our needs.

<configure 
   xmlns="http://namespaces.zope.org/zope" 
   i18n_domain="todo" 
   package="todo" 
   > 

  <include package="todo" /> 
  <include package="todo_plus" /> 

  <!-- Typical functional testing security setup --> 
  <securityPolicy 
      component="zope.securitypolicy.zopepolicy.ZopeSecurityPolicy" 
      /> 

  <unauthenticatedPrincipal 
      id="zope.anybody" 
      title="Unauthenticated User" 
      /> 
  <grant 
      permission="zope.View" 
      principal="zope.anybody" 
      /> 

  <principal 
      id="zope.mgr" 
      title="Manager" 
      login="mgr" 
      password="mgrpw" 
      /> 

  <role id="zope.Manager" title="Site Manager" /> 
  <grantAll role="zope.Manager" /> 
  <grant role="zope.Manager" principal="zope.mgr" /> 

As shown in the preceding code, the configuration simply includes a security policy, complete with users and roles and the packages that should be loaded by the instance, in addition to the regular Grok infrastructure. If we run any tests that require an authenticated user to work, we'll use these special users.

The includes at the top of the file just make sure that all of the Zope Component Architecture setup needed by our application is performed prior to running the tests.

tests.py

The default test module is very simple. It defines the functional layer and registers the tests for our package:

import os.path 
import z3c.testsetup 
import todo 
from zope.app.testing.functional import ZCMLLayer 

ftesting_zcml = os.path.join( 
    os.path.dirname(todo.__file__), 'ftesting.zcml') 
FunctionalLayer = ZCMLLayer(ftesting_zcml, __name__, 'FunctionalLayer', allow_teardown=True) 

test_suite = z3c.testsetup.register_all_tests('todo')

After the imports, the first line gets the path for the ftesting.zcml file, which then is passed to the layer definition method ZCMLLayer. The final line in the module tells the test runner to find and register all of the tests in the package.

This will be enough for our testing needs in this article, but if we needed to create another non-Grok package for our application, we would need to add a line like the last one to it, so that all of its tests are found by the test runner. This is pretty much boilerplate code, as only the package name has to be changed.

Create flexible, agile web applications using the power of Grok—a Python web framework Develop efficient and powerful web applications and web sites from start to finish using Grok, which is based on Zope 3 Integrate your applications or web sites with relational databases easily Extend your applications using the power of the Zope Toolkit Easy-to-follow and packed with practical, working code with clear explanations  http://www.packtpub.com/grok-1-0-web-development/book

app.txt

We finally come to the reason for this entire configuration—the actual tests that will be executed by the test runner. By default, the tests are included inside the app.txt file:

Do a functional doctest test on the app. 
======================================== 

:doctest:
:layer: todo.tests.FunctionalLayer 

Let's first create an instance of Todo at the top level: 

   >>> from todo.app import Todo 
   >>> root = getRootFolder() 
   >>> root['app'] = Todo() 

Run tests in the testbrowser 
---------------------------- 

The zope.testbrowser.browser module exposes a Browser class that 
simulates a web browser similar to Mozilla Firefox or IE. We use that 
to test how our application behaves in a browser. For more 
information, see http://pypi.python.org/pypi/zope.testbrowser. 

Create a browser and visit the instance you just created: 

   >>> from zope.testbrowser.testing import Browser 
   >>> browser = Browser() 
   >>> browser.open('http://localhost/app') 

Check some basic information about the page you visit: 

   >>> browser.url 
   'http://localhost/app/@@login?camefrom=/app/@@index' 
   >>> browser.headers.get('Status').upper() 
   '200 OK'

The text file has a title and immediately after that a :doctest: declaration—a declaration which tells the test runner that these tests need a functional layer to be loaded for their execution. Then comes a :layer: declaration, which is a path that points to the layer that we defined earlier in tests.py. After that, comes the test code. Lines starting with three brackets represent the Python code that is tested. Anything else is commentary.

When using the Python interpreter, a line of code may return a value, in which case, the expected return value must be written immediately below that line. This expected value will be compared with the real return value of the tested code and a failure will be reported, if the values don't match. Similarly, a line which is followed by an empty line will produce a failure, when the code is executed and a result is returned, because it is assumed that the expected return value in that case is None.

For example, in the last line of the Python doctest, the expression browser.headers.get('Status').upper() is expected to return the value 200 OK. If anything else is returned, the test will fail, even if there's just a slight difference.

Adding our own tests

Now, let's add a few functional tests that are specific to our application. We will need to emulate a browser for that. The zope.testbrowser package includes a browser emulator. We can pass any valid URL to this browser by using browser.open, and it will send a request to our application exactly like a browser would. The response from our application will be then available as browser.contents, so that we can perform our testing comparisons on it.

The Browser class

Before writing our tests, it will be useful to see what exactly our testbrowser can do. Of course, anything that depends on JavaScript will not work here, but other than that, we can interact with links and even forms in a very straightforward manner. Here's a look at the main functionality offered by the Browser class:

Now that we know what we can do, let's try our hand at writing some tests.

Our first to-do application tests

Ideally, we should have been adding a couple of doctests to the app.txt file every time we add a new functionality to our application. We have gone through the reasons why we didn't do so, but let's recover some lost ground. At the very least, we'll get a feeling of how doctests work.

We'll add our new tests to the existing app.txt file. The last test there that we saw left us at the to-do instance URL. We are not logged in, so if we print the browser contents, we will get the login page. Let's add a test for this:

Since we haven't logged in, we can't see the application. The login
page appears:

    >>> 'Username' in browser.contents
    True
    >>> 'Password' in browser.contents 
    True 

As we mentioned earlier, when visiting a URL with the testbrowser, the entire HTML content of the page is stored in browser.contents. Now we know that our login page has a username and a password field, so we simply use a couple of in expressions and check if these fields evaluate to True. If they do, it would mean that the browser is effectively looking at the login page.

Let's add a test for logging in. When we start the application in the tests, the user database is empty, therefore, the most economical way of logging in is to use basic authentication. This can be easily done by changing the request headers:

 >>> browser.addHeader('Authorization', 'Basic mgr:mgrpw') 
 >>> browser.open('http://localhost/app') 
 >>> 'Logged in as Manager' in browser.contents 
 True 

That's it. We just add the header, "reload" the home page, and we should be logged in. We verify it by looking for the Logged in as message, which we know has to be there after a successful login.

Once we are logged in, we can finally test our application properly. Let's begin by adding a project:

We are now in. Let's create a project:

    >>> browser.getLink('Create a new project').click() 
    >>> browser.getControl(name='form.title').value='a project' 
    >>> browser.getControl(name='form.description').value='The 
description.' 
    >>> browser.getControl('Add project').click() 
    >>> browser.url 
    'http://localhost/app/0' 
    >>> 'Create new list' in browser.contents 
    True 

First, we find the link on the home page that will take us to the 'add form' project. This is done easily with the help of the getLink method and the text of the link. We click on the link and now should have the form ready to fill in. We then use getControl to find each field by its name and change its value. Finally, we submit the form by getting the submit button control and clicking on it. The result is that the project is created and we are redirected to its main view. We can confirm this by comparing the browser url with the URL that we would expect in this case.

Adding a list to the project is just as easy. We get the form controls, assign them some values, and click on the submit button. The list and the link for adding new items to it should appear in the browser contents:

We have added a project. Now, we'll add a list to it. If we are successful, we will seea link for adding a new item for the list:

    >>> browser.getControl(name='title').value='a list' 
    >>> browser.getControl(name='description').value='The list 
description.' 
    >>> browser.getControl(name='new_list').click() 
    >>> 'New item' in browser.contents 
    True 

Good. Let's see how we are doing so far:

$ bin/testRunning tests at level 1 
Running todo.FunctionalLayer tests: 
  Set up 
 in 3.087 seconds. 
  Running: 
.......2009-09-30 21:35:44,585 INFO sqlalchemy.engine.base.
Engine.0x...69ec PRAGMA table_info("users") 
2009-09-30 21:35:44,585 INFO sqlalchemy.engine.base.Engine.0x...69ec () 

  Ran 7 tests with 0 failures and 0 errors in 0.428 seconds. 
Tearing down left over layers: 
  Tear down todo.FunctionalLayer ... not supported 

Not bad. We now have four more working tests than when we started.

Note that the test browser handles HTTP errors gracefully, returning a string similar to what a real browser would show when running into an error. For example, take a look at the following test:

>>> browser.open('http://localhost/invalid')
Traceback (most recent call last):
...
HTTPError: HTTP Error 404: Not Found

That's the default behavior because this is how real browsers work, but sometimes, when we are debugging, it's better to take a look at the original exception caused by our application. In such a case, we can make the browser stop handling errors automatically and throw the original exceptions, so that we can handle them. This is done by setting the browser.handleErrors property to False:

>>> browser.handleErrors = False
>>> browser.open('http://localhost/invalid')
Traceback (most recent call last):
...
NotFound: Object: <zope.site.folder.Folder object at ...>,
          name: u'invalid'

[ 1 | 2 ]

---

If you have read this article you may be interested to view :

• Testing and Debugging in Grok 1.0: Part 2
• Forms in Grok 1.0

---

Grok 1.0 Web Development

 

Create flexible, agile web applications using the power of Grok—a Python web framework Develop efficient and powerful web applications and web sites from start to finish using Grok, which is based on Zope 3 Integrate your applications or web sites with relational databases easily Extend your applications using the power of the Zope Toolkit Easy-to-follow and packed with practical, working code with clear explanations  http://www.packtpub.com/grok-1-0-web-development/book

---

About the author

Carlos de la Guardia

Carlos has been doing web consulting and development since 1994, when selling any kind of project required two meetings just to explain what the Internet was in the first place. He was the co-founder of Aldea Systems, a web consulting company where he spent ten years working in all kinds of web projects using a diverse range of languages and tools. In 2005 he became an independent developer and consultant, specializing in Zope and Plone projects. He frequently blogs about Plone and other Zope-related subjects.

---

Books from Packt

Joomla! 1.5 Multimedia

TYPO3 4.3 Multimedia Cookbook

jQuery 1.4 Reference Guide

Python Testing: Beginner's Guide

Backbase 4 RIA Development

AJAX and PHP: Building Modern Web Applications 2nd Edition

jQuery UI 1.7: The User Interface Library for jQuery

CodeIgniter 1.7

---

---

---