Chapter 7. If Ain't Tested, It Ain't Game, Bro!

Does the software you write have quality? How do you attest that?

Software is usually written according to certain requested needs, be it bug reports, feature and enhancement tickets, or whatever. To have quality, the software must satisfy these needs wholly and precisely; that is, it should do what is expected of it.

Just as you would push a button to know what it does (given you do not have a manual), you have to test your code to know what it does or to attest what it should do. That's how you assure software quality.

During the course of a software development, it is usual to have many features that share some code base or library. You could, for example, change a piece of code to fix a bug and create another bug in another point in your code. Software tests also help with that as they assure that your code does what it should do; if you change a piece of broken code and break another piece of code, you'll also be breaking a test. In this...

What kinds of test are there?

We want tests, and we want them now; but what kind of test do we want?

There are two major classifications for tests, based on how much access to the internal code you have: black-box and white-box tests.

Black-box tests are where the testers do not have knowledge of, and/or access to, the actual code he/she is testing. In these cases, the test consists of checking whether the system states before and after the code execution are as expected or whether the given output corresponds to the given input.

White-box tests are a little different as you will have access to the actual code internals that you're testing as well as the system expected states before and after code execution and the expected output for a given input. This kind of test has a stronger subjective goal, usually related to performance and software quality.

In this chapter, we will cover how to implement black-box tests as they are more accessible to others and easier to implement. On the other hand...

Fixtures

Good tests are always executed considering a predefined, reproducible application state; that is, whenever you run a test in the chosen state, the result will always be equivalent. Usually, this is achieved by setting your database data yourself and clearing your cache and any temporary files (if you make use of external services, you should mock them) for each test. Clearing cache and temporary files is not hard, while setting your database data, on the other hand, is.

If you're using Flask-SQLAlchemy to hold your data, you would need to hardcode, somewhere in your tests as follows:

attributes = { … }
model = MyModel(**attributes)
db.session.add(model)
db.session.commit()

This approach does not scale as it is not easily reusable (when you define this as a function and a method, define it for each test). There are two ways to populate your database for testing: fixtures and pseudo-random data.

Using pseudo-random data is usually library-specific and produces better test data as the...

Summary

Whoa! We just survived a chapter about testing software! That's something to be proud of. We learned a few concepts such as TDD, white-box, and black-box testing. We also learned how to create unit tests; test our views; write features using the Gherkin language and test them using lettuce; use Flask-testing, Selenium with PhantomJS to test a HTML response from the user perspective; also how to use fixtures to control our application state for proper reproducible testing. Now you are capable of testing Flask applications in different ways using the correct techniques for different scenarios and needs.

In the next chapter, things are gonna go wild really fast as our subject of study will be tricks with Flask. Blueprints, sessions, logging, debugging, and so on, will be covered in the next chapter, allowing you to create even more robust software. See you there!