Chapter 6. Profiling, Debugging, and Testing

Profiling, debugging, and testing are an integral part of the development process. You are probably familiar with the concept of unit testing. Unit tests are automated tests written by a programmer to test his or her code. These tests could, for example, test a function or part of a function in isolation. Only a small unit of code is tested in each test. The benefits are increased confidence in the quality of the code, reproducible tests and, as a side effect, more clear and correct code. Unit testing also facilitates collaborative editing because, usually, no one understands all the code in a complex project themselves, so unit tests prevent contributors from breaking the existing code. Python has good support for unit testing. NumPy adds the numpy.testing package to help NumPy code the unit testing.

This chapter's topics include:

• Asserts

• Profiling

• Debugging

• Unit testing

Assert functions

The NumPy testing package has a number of utility functions that test whether a precondition is true or not. The following table lists the NumPy assert functions:

Profiling a program with IPython

As most of us learned in programming classes, premature optimization is the root of all evil. However, once you approach the final stages of software development, it could very well be that certain parts of the code are unnecessarily slow or use more memory than is strictly needed. We can find these issues through the process of profiling. Profiling involves measuring metrics such as execution time for a piece of code such as a function or a single statement.

IPython is an interactive Python environment, which also comes with a shell similar to the standard Python shell. In IPython, we can profile small snippets of code using timeit. We can also profile a larger script. We will show both approaches.

1. Timing a snippet:

   Start IPython in pylab mode

   ipython -pylab
   

2. Create an array containing 1,000 integer values between 0 and 1,000.

   In [1]: a = arange(1000)
   

   This is the time to search for the answer to everything 42 in the array.

   In [2]: %timeit searchsorted(a, 42...

Debugging with IPython

Debugging is one of those tasks that we try to avoid by having good unit tests in place. Debugging can take a long time, and most likely, you don't have that time. Therefore, it is important to be systematic and know your tools well. After you have found the issue and implemented a fix, you should have a unit test in place. This way at least you will not have to go through the torture of debugging again.

We will debug some incorrect code that is trying to access an array element out of bounds:

import numpy

a = numpy.arange(7)
print a[8]

Proceed with the following steps:

1. Run the faulty script in IPython.

   Start the ipython shell. Run the faulty script in IPython by issuing the following command:

   In [1]: %run buggy.py
   ---------------------------------------------------------------------------
   IndexError                                Traceback (most recent call last)
   .../site-packages/IPython/utils/py3compat.pyc in execfile(fname, *where)
       173             else:
       174...

Performing Unit tests

Unit tests are automated tests that test a small piece of code, usually a function or method. Python has the PyUnit API for unit testing. As NumPy users, we can make use of the assert functions that we saw in action before.

We will write tests for a simple factorial function. The tests will check for the so-called happy path (regular conditions and is expected to always pass) and for abnormal conditions:

1. We start by writing the factorial function:

   def factorial(n):
      if n == 0:
         return 1
   
      if n < 0:
         raise ValueError, "Unexpected negative value"
   
      return np.arange(1, n+1).cumprod()

   The code is using the arange and cumprod functions that we have already seen to create arrays and calculate the cumulative product, but we added a few checks for boundary conditions.

2. Now we will write the unit test. Let's write a class that will contain the unit tests. It extends the TestCase class from the unittest module, which is a part of standard Python. We test for calling...

Nose tests decorators

Nose is a Python framework that makes (unit) testing a bit easier. Nose helps you organize tests. According to the nose documentation:

Any python source file, directory or package that matches the testMatch regular expression (by default: (?:^|[b_.-])[Tt]est) will be collected as a test.

Nose makes extensive use of decorators. Python decorators are annotations that indicate something about a method or a function. The numpy.testing module has a number of decorators:

Function	Description
assert_almost_equal	This raises an exception if two numbers are not equal up to a specified precision
assert_approx_equal	This raises an exception if two numbers are not equal up to a certain significance
assert_array_almost_equal	This raises an exception if two arrays are not equal up to a specified precision
assert_array_equal	This raises an exception if two arrays are not equal
assert_array_less	This raises an exception if two arrays do not have the same shape, and the elements of the first array are strictly less than the elements of the second array
assert_equal	This raises an exception if two objects are not equal
assert_raises	This fails if a specified exception is not raised by a callable function invoked with defined arguments
assert_warns	This fails...

Decorator	Description
numpy.testing.decorators.deprecated	This is the filter's deprecation warning when running tests
numpy.testing.decorators.knownfailureif	This raises the KnownFailureTest exception based on a condition.
numpy.testing.decorators.setastest	This marks a function as being a test or not being a test.
numpy.testing.decorators.skipif	This raises the SkipTest exception based on a condition.
numpy.testing.decorators.slow	This labels test functions or methods as slow.

Additionally...

Summary

We learned about testing and NumPy testing utilities in this chapter. We covered unit testing, assert functions, profiling, and debugging. Unit testing is a standard practice since it should give you better quality code with a low risk of regression. NumPy provides assert functions to help you with unit testing. We covered some of these functions in this chapter. No matter how good your unit tests are, at a certain point, you will have to do profiling and debugging, so pointers are given in that respect.

The topic of the next chapter is the scientific Python ecosystem and how NumPy fits in it.

The rest of the chapter is locked

Profiling a program with IPython

As most of us learned in programming classes, premature optimization is the root of all evil. However, once you approach the final stages of software development, it could very well be that certain parts of the code are unnecessarily slow or use more memory than is strictly needed. We can find these issues through the process of profiling. Profiling involves measuring metrics such as execution time for a piece of code such as a function or a single statement.

IPython is an interactive Python environment, which also comes with a shell similar to the standard Python shell. In IPython, we can profile small snippets of code using timeit. We can also profile a larger script. We will show both approaches.

1. Timing a snippet:

   Start IPython in pylab mode

   ipython -pylab
   

2. Create an array containing 1,000 integer values between 0 and 1,000.

   In [1]: a = arange(1000)
   

   This is the time to search for the answer to everything 42 in the array.

   In [2]: %timeit searchsorted(a, 42...

Debugging with IPython

Debugging is one of those tasks that we try to avoid by having good unit tests in place. Debugging can take a long time, and most likely, you don't have that time. Therefore, it is important to be systematic and know your tools well. After you have found the issue and implemented a fix, you should have a unit test in place. This way at least you will not have to go through the torture of debugging again.

We will debug some incorrect code that is trying to access an array element out of bounds:

import numpy

a = numpy.arange(7)
print a[8]

Proceed with the following steps:

1. Run the faulty script in IPython.

   Start the ipython shell. Run the faulty script in IPython by issuing the following command:

   In [1]: %run buggy.py
   ---------------------------------------------------------------------------
   IndexError                                Traceback (most recent call last)
   .../site-packages/IPython/utils/py3compat.pyc in execfile(fname, *where)
       173             else:
       174...

Performing Unit tests

Unit tests are automated tests that test a small piece of code, usually a function or method. Python has the PyUnit API for unit testing. As NumPy users, we can make use of the assert functions that we saw in action before.

We will write tests for a simple factorial function. The tests will check for the so-called happy path (regular conditions and is expected to always pass) and for abnormal conditions:

1. We start by writing the factorial function:

   def factorial(n):
      if n == 0:
         return 1
   
      if n < 0:
         raise ValueError, "Unexpected negative value"
   
      return np.arange(1, n+1).cumprod()

   The code is using the arange and cumprod functions that we have already seen to create arrays and calculate the cumulative product, but we added a few checks for boundary conditions.

2. Now we will write the unit test. Let's write a class that will contain the unit tests. It extends the TestCase class from the unittest module, which is a part of standard Python. We test for calling...

Nose tests decorators

Nose is a Python framework that makes (unit) testing a bit easier. Nose helps you organize tests. According to the nose documentation:

Any python source file, directory or package that matches the testMatch regular expression (by default: (?:^|[b_.-])[Tt]est) will be collected as a test.

Nose makes extensive use of decorators. Python decorators are annotations that indicate something about a method or a function. The numpy.testing module has a number of decorators:

Summary

We learned about testing and NumPy testing utilities in this chapter. We covered unit testing, assert functions, profiling, and debugging. Unit testing is a standard practice since it should give you better quality code with a low risk of regression. NumPy provides assert functions to help you with unit testing. We covered some of these functions in this chapter. No matter how good your unit tests are, at a certain point, you will have to do profiling and debugging, so pointers are given in that respect.

The topic of the next chapter is the scientific Python ecosystem and how NumPy fits in it.