You're a programmer: a coder, a developer, or maybe a hacker! As such, it's almost impossible that you haven't had to sit down with a program that you were sure was ready for use—or worse yet, a program you knew was not ready—and put together a bunch of tests to prove it. It often feels like an exercise in futility, or at best a waste of time. We'll learn how to avoid that situation, and make testing an easy and enjoyable process.
This book is going to show you a new way to test, a way that puts much of the burden of testing right where it should be: on the computer. Even better, your tests will help you to find problems early and tell you just where they are, so that you can fix them easily. You'll love the easy, helpful methods of automated testing and test-driven development that you will learn about in this book.
The Python language has some of the best tools available, when it comes to testing. As a result, we'll learn how to make testing something that is easy, quick, and fun by taking advantage of those tools.
In this book, we'll:
Study popular testing tools such as doctest, unittest, and Nose
Learn about testing philosophies like unit testing and test-driven development
Examine the use of mock objects and other useful testing secrets
Learn how to integrate testing with the other tools that we use, and with our workflow
Introduce some secondary tools that make it easier to use the major testing tools
This chapter started with a lot of grandiose claims, such as: You'll enjoy testing. You'll rely on it to help you kill bugs early and easily. Testing will stop being a burden for you, and become something that you want to do. You may be wondering how this is possible?
Think back to the last annoying bug that you had to deal with. It could have been anything; a database schema mismatch, or a bad data structure.
Remember what caused the bug? The one line of code with a subtle logic error? The function that didn't do what the documents said it would do? Whatever it was, keep it in mind.
Imagine a small chunk of code that could have caught the bug, if it had been run at the right time, and informed you about it.
Now imagine that all of your code was accompanied by those little chunks of test code, and that they are quick and easy to execute.
How long would your bug have survived? Not very long at all.
That gives you a basic understanding of what we'll be talking about in this book. There are many tools and refinements that can make the process quicker and easier. The basic idea is to tell the computer what you expect, using simple and easily-written chunks of code, and then have the computer double-check your expectations throughout the coding process. As expectations are easy to describe, you can write them down first, allowing the computer to shoulder much of the burden of debugging your code. As a result, you can move on to interesting things while the computer keeps a track of everything else.
When you're done, you'll have a code base that is highly tested and that you can be confident in. You will have caught your bugs early and fixed them quickly. The best part is that your testing was done by the computer based on what you told it you wanted the program to do. After all, why should you do it, when the computer can do it for you?
I have programmed simple automated tests to catch everything from minor typos, to instances of database access code being left dangerously out of date after a schema change, and pretty much any other bug you can imagine. The tests caught the errors quickly, and pinpointed their locations. A great deal of effort and bother was avoided because they were there.
Imagine the time that you'll save or spend on writing new features, instead of chasing old bugs. Better code, written more quickly, has a good cost/benefit ratio. Testing the right way really is both more fun and more profitable.
Testing is commonly divided into several categories, based on how complex the component being tested is. Most of our time will be focused on the lowest level—unit testing—because tests in the other categories operate on pretty much the same principles.
Unit testing is testing of the smallest possible pieces of a program. Often, this means individual functions or methods. The keyword here is individual; something is a unit if it there's no meaningful way to divide it up further.
Unit tests are used to test a single unit in isolation, verifying that it works as expected, without considering what the rest of the program would do. This protects each unit from inheriting bugs from mistakes made elsewhere, and makes it easy to narrow down on the actual problem.
By itself, unit testing isn't enough to confirm that a complete program works correctly, but it's the foundation upon which everything else is based. You can't build a house without solid materials, and you can't build a program without units that work as expected!
In integration testing, the boundaries of isolation are pushed further back, so that the tests encompass interactions between related units. Each test should still be run in isolation, to avoid inheriting problems from outside, but now the test checks whether the tested units behave correctly as a group.
Integration testing can be performed with the same tools as unit testing. For this reason, newcomers to automated testing are sometimes lured into ignoring the distinction between unit testing and integration testing. Ignoring this distinction is dangerous, because such multipurpose tests often make assumptions about the correctness of some of the units that they involve. This means that the tester loses much of the benefit which automated testing would have granted. We're not aware of the assumptions we make until they bite us, so we need to consciously choose to work in a way that minimizes assumptions. That's one of the reasons why I refer to test-driven development as a discipline.
System testing extends the boundaries of isolation even further, to the point where they don't even exist. System tests check parts of the program, after the whole thing has been plugged together. In a sense, system tests are an extreme form of integration tests.
System tests are very important, but they're not very useful without integration tests and unit tests. You have to be sure of the pieces before you can be sure of the whole. If there's a subtle error somewhere, system testing will tell you that it exists, but not where it is or how to fix it. The odds are good that you've experienced that situation before; it's probably why you hate testing.
This book assumes that you have working knowledge of the Python programming language, and that you have a fully functional Python interpreter available. The assumption is that you have at least version 2.6 of Python, which you can download from http://www.python.org/. If you have an earlier version, don't worry: there are sidebars that will help you navigate the differences. You'll also need your favorite text editor.
In this chapter, we learned what this book is about and what to expect from it. We took a glance at the philosophy of automated testing and test-driven development.
We talked about the different types of tests that combine together to form a complete suite of tests for a program, namely: unit tests, integration tests, and system tests. We learned that unit tests are related to the fundamental components of a program (such as functions), integration tests cover larger swaths of a program (like modules), and system tests encompass testing a program in its entirety.
We learned about how automated testing can help us, by moving the burden of testing mostly onto the computer. You can tell the computer how to check your code, instead of having to do the checks for yourself. That makes it convenient to check your code earlier and more often, saves you from overlooking the things that you would otherwise miss, and helps you quickly locate and fix bugs.
We shed some light on test-driven development, the discipline of writing your tests first, and letting them tell you what needs to be done, in order to write the code you need.
We also discussed the development environment that you'll need, in order to work through this book.
Now that we've learned about the lay of the land (so to speak), we're ready to start writing tests—which is the topic of the next chapter.
