Developing an Application Test Harness

In the previous chapter, we developed several modules that will be used in an application project to perform I/O functionality and interact with a user. The question, though, is how can we be certain that those modules we created actually work? We created several tests, but if we make any changes to them, we have to rerun those tests manually. By doing this, we'll find that it is easy to overlook a potential bug. Manual testing is not just time-consuming but also error-prone.

In this chapter, we are going to develop an application test harness that we can use to test our MicroPython modules and ensure they have the fewest number of bugs.

The following topics will be covered in this chapter:

• What a test harness is and why we would want to use one
• Requirements for developing a test harness
• Test harness design
• Test harness construction...

Technical requirements

The example code for this chapter can be found in this book's GitHub repository: https://github.com/PacktPublishing/MicroPython-Projects/tree/master/Chapter04.

To run the examples and your own test harness, you will need the following hardware and software:

• Pyboard Revision 1.0 or 1.1
• RobotDyn I2C 8-bit PCA8574 I/O expander module or equivalent
• Adafruit RGB Pushbutton PN: 3423 or equivalent
• A terminal application (PuTTy, RealTerm, or Terminal)
• A text editor such as Sublime Text

A brief introduction to test harnesses

A test harness is a collection of software and data that is used to automatically test application modules under various conditions in order to determine whether they meet the design requirements. A test harness will often consist of three main components, as follows:

• A test execution engine: This is a piece of software that interfaces with the application modules under test and provides them with various inputs. After doing this, it monitors their outputs to ensure that the expected result is achieved. The test execution engine is usually written in the same language as the application modules that are under test.
• A repository of tests: These are additional software modules that are written that contain the desired conditions under which the modules will be tested. The tests also contain the expected output for those tests so that it can...

Test harness requirements

There are several areas where we need to make careful considerations for our test harness. First, we need to consider the hardware requirements. When we develop our test harness, we may need to design and implement a hardware interface that will allow us to not just interact with the system but also verify various communication and control aspects of the software. This requires us to clearly define any additional hardware that will be required.

Second, we need to define what our software requirements will be. These include areas such as how the test harness will be executed, what's needed when it comes to reporting, and the language that will be used to execute our tests. In this section, we will explore these requirements and put together the requirements for the test harness that we will build to test our MicroPython code. Keep in mind that we...

Test harness design

At this stage in the project, we've discovered what the basic requirements for our test harness are going to be. Now, we are ready to dive deeper into the hardware and software architecture that is necessary to support the test harness. Just like in previous chapters, at this stage, we are going to start with an architecture that will then feed into a more detailed design that is then used in the construction phase. Remember, the architecture should be flexible so that we can deal with any changing requirements on the fly.

The test harness hardware architecture

The hardware architecture for our project requires just a few minor adjustments in order to support our test harness. These updates will provide...

Constructing the test harness

The bulk of the test harness that we will be constructing is going to be software. The modules that we are going to build the test harness for are the modules that we created in Chapter 3, Writing a MicroPython Driver for an I/O Expander. We could build a harness that not only includes low-level module testing but also the high-level system behavior; however, I want to convey how we can create a test harness. I will leave the high-level application testing to you as an exercise. The modules that we will be testing are as follows:

• PCA8574.py
• LED_RGB.py
• Button_rgb.py

Let's get started!

Writing the test harness scaffolding

Before we dive in and start developing tests, we should spend a few...

Running the test harness

Running the test harness on the target will not be any different than running any other MicroPython script. Copy main.py, test_harness, and the test scripts to the pyboard. From the terminal, you can use Ctrl + C to terminate any application that is already running. Then, use Ctrl + D to cause a soft reset to occur on the pyboard. At this point, you should see an output from the test harness similar to the following:

From the preceding output, we can see that the script has been started and that testing has begun. We can see our test setup operation followed by the execution and reporting operations. We can also see that the first three tests pass successfully but that our final test, the LSB I/O – Low test, fails. From the test report, it looks like the I/O stayed at the high state rather than toggling low.

There are several potential causes for...

Summary

In this chapter, we explored the different methods that we can use to create a test harness for MicroPython modules and application code. While there are many methods available, ranging from very simple to highly sophisticated, we implemented a simple test harness that you can easily leverage and use as the foundation to build even more useful harnesses. The harness that we built used the device under test to simulate system inputs, which allowed us to keep the test harness' costs low and required no additional external hardware.

In the next chapter, we are going to dive deep into the MicroPython kernel and learn how we can customize the startup code and the kernel for our own application needs. As part of this project, we are going to compile our own kernel and deploy it to a development board that doesn't come with MicroPython installed on it by default.

...

Questions

1. What are the three main components that are part of nearly every test harness?
2. What are the advantages of using a test harness?
3. What are a few examples of faults that we would want a test harness to test for?
4. What are some of the architectures that a test harness can follow?
5. What are the four operations that we need our module tests to perform?

Further reading

• Test-Driven Development with Python, Harry Percival
• Test-Driven Development for Embedded C, James W. Grenning