Error and Exception Handling

In this chapter, we will cover errors and exceptions and how to find and fix them. Handling exceptions is an important part of writing reliable and usable code. We will introduce basic built-in exceptions and show how to use and treat exceptions. We'll introduce debugging and show you how to use the built-in Python debugger.

In this chapter, we cover the following topics:

• What are exceptions?
• Finding errors: debugging

12.1 What are exceptions?

The first error that programmers (even experienced ones) are confronted with is when the code has incorrect syntax, meaning that the code instructions are not correctly formatted.

Consider this example of a syntax error:

>>> for i in range(10)
  File “<stdin>”, line 1
    for i in range(10)
                      ^
SyntaxError: invalid syntax

The error occurs because of a missing colon at the end of the for declaration. This is an example of an exception being raised. In the case of SyntaxError, it tells the programmer that the code has incorrect syntax and also prints the line where the error occurred, with an arrow pointing to where in that line the problem is.

Exceptions in Python are derived (inherited) from a base class called Exception. Python comes with a number of built-in exceptions. Some common exception types are listed in Table 12.1.

Here are two common examples of exceptions...

12.1.1 Basic principles

Let's look at the basic principles on how to use exceptions by raising them with raise and catching them with try statements.

Raising exceptions

Creating an error is referred to as raising an exception. You saw some examples of exceptions in the previous section. You can also define your own exceptions of a predefined type or use an exception of an unspecified type. Raising an exception is done with a command like this:

raise Exception("Something went wrong")

Here an exception of an unspecified type was raised.

It might be tempting to print out error messages when something goes wrong, for example, like this:

print("The algorithm did not converge.")

This is not recommended for a number of reasons. Firstly, printouts are easy to miss, especially if the message is buried in many other messages being printed to your console. Secondly, and more importantly, it renders your code unusable by other code. The calling code will not read what you printed and will not have a way of knowing that an error occurred and therefore has no way of taking care of it.

For these reasons, it is always better...

Catching exceptions

Dealing with an exception is referred to as catching an exception. Checking for exceptions is done with the commands try and except.

An exception stops the program execution flow and looks for the closest try enclosing block. If the exception is not caught, the program unit is left and it continues searching for the next enclosing try block in a program unit higher up in the calling stack. If no block is found and the exception is not handled, execution stops entirely and the standard traceback information is displayed.

Let's look at the factorial example from previously and use it with the try statement:

n=-3
try:
    print(factorial(n))
except ValueError:
    print(factorial(-n))   # Here we catch the error

In this case, if the code inside the try block raises an error of type ValueError, the exception will be caught and the action in the except block is taken. If no exception occurs inside...

12.1.2 User-defined exceptions

Besides the built-in Python exceptions, it is also possible to define your own exceptions. Such user-defined exceptions should inherit from the base class Exception. This can be useful when you define your own classes such as the polynomial class in Section 19.1.

Take a look at this small example of a simple user-defined exception:

class MyError(Exception):
    def __init__(self, expr):
        self.expr = expr
    def __str__(self):
        return str(self.expr)

try:
   x = random.rand()
   if x < 0.5:
      raise MyError(x)
except MyError as e:
   print("Random number too small", e.expr)
else:
   print(x)

A random number is generated. If the number is below 0.5, an exception is thrown and a message that the value is too small is printed. If no exception is raised, the number is printed.

In this example, you also saw a case of using else in a try statement. The block under else ...

12.1.3 Context managers – the with statement

There is a very useful construction in Python for simplifying exception handling when working with contexts such as files or databases. The statement encapsulates the structure try ... finally in one simple command. Here is an example of using with to read a file:

with open('data.txt', 'w') as f:
    process_file_data(f)

This will try to open the file, run the specified operations on the file (for example, reading), and close the file. If anything goes wrong during the execution of process_file_data, the file is closed properly and then the exception is raised. This is equivalent to:

f = open('data.txt', 'w')
try: 
    # some function that does something with the file 
    process_file_data(f) 
except:
    ... 
finally:
    f.close()

We will use this option in Section 14.1: File handling, when reading and writing files.

The preceding file-reading...

12.2 Finding errors: debugging

Errors in software code are sometimes referred to as bugs. Debugging is the process of finding and fixing bugs in code. This process can be performed at varying degrees of sophistication. The most efficient way is to use a tool called a debugger. Having unit tests in place is a good way to identify errors early; see Section 15.2.2: Using the unittest package. When it is not obvious where or what the problem is, a debugger is very useful.

12.2.1 Bugs

There are typically two kinds of bugs:

• An exception is raised and not caught.
• The code does not function properly.

The first case is usually easier to fix. The second can be more difficult as the problem can be a faulty idea or solution, a faulty implementation, or a combination of the two.

We are only concerned with the first case in what follows, but the same tools can be used to help find why the code does not do what it is supposed to.

12.2.2 The stack

When an exception is raised, you see the call stack. The call stack contains the trace of all the functions that called the code where the exception was raised.

A simple stack example is:

def f():
   g()
def g():
   h()
def h():
   1//0

f()

The stack, in this case, is f, g, and h. The output generated by running this piece of code looks like this:

Traceback (most recent call last):
  File "stack_example.py", line 11, in <module>
    f() 
  File "stack_example.py", line 3, in f
    g() 
  File "stack_example.py", line 6, in g
    h() File "stack_example.py", line 9, in h
    1//0 
ZeroDivisionError: integer division or modulo by zero

The error is printed. The sequence of functions leading up to the error is shown. The function f on line 11 was called, which in turn called g and then h. This caused ZeroDivisionError.

A stack trace reports on the active stack at a certain...

12.2.3 The Python debugger

Python comes with its own built-in debugger called pdb. Some development environments come with the debugger integrated. The following process still holds in most of these cases.

The easiest way to use the debugger is to enable stack tracing at the point in your code that you want to investigate. Here is a simple example of triggering the debugger based on the example mentioned in Section 7.3: Return values:

import pdb

def complex_to_polar(z):
    pdb.set_trace() 
    r = sqrt(z.real ** 2 + z.imag ** 2)
    phi = arctan2(z.imag, z.real)
    return (r,phi)
z = 3 + 5j 
r,phi = complex_to_polar(z)

print(r,phi)

The command pdb.set_trace() starts the debugger and enables the tracing of subsequent commands. The preceding code will show this:

> debugging_example.py(7)complex_to_polar()
-> r = sqrt(z.real ** 2 + z.imag ** 2) 
(Pdb)

The debugger prompt is indicated with (Pdb). The debugger stops the program execution and...

12.2.4 Overview – debug commands

In Table 12.2, the most common debug commands are shown. For a full listing and description of commands, see the documentation for more information [24]. Note that any Python command also works, for example, assigning values to variables.

If you want to inspect a variable with a name that coincides with any of the debugger's short commands, for example, h, you must use !h to display the variable.

Command	Action
h	Help (without arguments, it prints available commands)
l	Lists the code around the current line
q	Quit (exits the debugger and the execution stops)
c	Continues execution
r	Continues execution until the current function returns
n	Continues execution until the next line
p <expression>	Evaluates and prints the expression in the current context

Table 12.2: The most common debug commands for the debugger

12.2.5 Debugging in IPython

IPython comes with a version of the debugger called ipdb. At the time of writing this book, the differences to pdb are very minor but this may change.

There is a command in IPython, %pdb, that automatically turns on the debugger in case of an exception. This is very useful when experimenting with new ideas or code. An example of how to automatically turn on the debugger in IPython is:

In [1]: %pdb # this is a so - called IPython magic command 
Automatic pdb calling has been turned ON

In [2]: a = 10

In [3]: b = 0

In [4]: c = a/b
___________________________________________________________________
ZeroDivisionError                  Traceback (most recent call last) 
<ipython-input-4-72278c42f391> in <module>() 
—-> 1 c = a/b

ZeroDivisionError: integer division or modulo by zero 
> <ipython-input-4-72278c42f391>(1)<module>()
      -1 c = a/b
ipdb>

The IPython magic command %pdb at the...

12.3 Summary

The key concepts in this chapter were exceptions and errors. We showed how an exception is raised to be caught later in another program unit. You can define your own exceptions and equip them with messages and current values of given variables.

The code may return unexpected results without throwing an exception. The technique to localize the source of the erroneous result is called debugging. We introduced debugging methods and hopefully encouraged you to train them so that you have them readily available when needed. The need for serious debugging comes sooner than you might expect.