Advanced IoT Programming Concepts - Threads, AsyncIO, and Event Loops

In the previous chapter, we learned how to detect movement with a PIR sensor, as well as measure distances and detect movement with ultrasonic sensors and Hall-effect sensors.

In this chapter, we will discuss alternative ways of structuring our Python programs when we are working with electronic sensors (input devices) and actuators (output devices). We will cover the classic event-loop approach to programming, before moving on to more advanced approaches, including the use of threads in Python, the publisher/subscriber model, and finally, asynchronous I/O programming with Python.

I guarantee you that there are many, many blog posts and tutorials across the internet covering these topics; however, what we will cover in this chapter will be uniquely focused on practical electronic interfacing. Our approach in...

Technical requirements

To perform the exercises in this chapter, you will need the following:

• Raspberry Pi 4 Model B
• Raspbian OS Buster (with desktop and recommended software)
• Minimum Python version 3.5

These requirements are what the code examples in this book are based on. It's reasonable to expect that the code examples should work without modification on Raspberry Pi 3 Model B or a different version of Raspbian OS as long as your Python version is 3.5 or higher.

You will find this chapter's source code in the chapter12 folder in the GitHub repository available at https://github.com/PacktPublishing/Practical-Python-Programming-for-IoT.

You will need to execute the following commands in a terminal to set up a virtual environment and install the Python libraries required for the code in this chapter:

$ cd chapter12              # Change into this chapter's folder
$ python3 -m venv venv      # Create Python Virtual Environment
$ source venv/bin/activate...

Building and testing our circuit

I'm going to present the circuit and programs for this chapter in the form of a practical exercise. Let's pretend for a moment that we have been asked to design and build a gizmo that has the following requirements:

• It has two LEDs that blink.
• A potentiometer is used to adjust the rate that the LED(s) blink.
• When the program starts, both LEDs will blink at the same rate determined by the position of the potentiometer.
• A blinking rate of 0 seconds means an LED is off, while the maximum blinking rate of 5 seconds means an LED is on for 5 seconds, then off for 5 seconds, before repeating the cycle.
• A push-button is used to select which LED changes its blinking rate when the potentiometer is adjusted.
• When the push-button is pressed and held for 0.5 seconds, all LEDs synchronize to the same rate, determined by the potentiometer's position.
• Ideally, the program code should easily scale to support more LEDs with minimal coding effort...

Building the reference circuit

In Figure 12.1 is a circuit that meets the requirements we just listed. It has a push-button, a potentiometer in the form of a voltage divider connected to an ADS1115 analog-to-digital converter, and two LEDs connected by current limiting resistors. Adding additional LEDs will be as simple as wiring more LED and resistors pairs between GND and a free GPIO pin:

Figure 12.1 – Reference circuit schematic

If you have not already created a similar circuit on your own, we will create this circuit now on your breadboard. We will build this circuit in three parts. Let's get started:

Figure 12.2 – Reference circuit (part 1 of 3)

Here are the steps to follow to create the first part of our breadboard build where we place the components. The step numbers match the numbers in black circles in Figure 12.2:

1. Place the ADS1115 module into your breadboard.
2. Place the potentiometer into your breadboard.
3. Place an LED into your breadboard, taking...

Running the examples

This chapter comes with four different versions of code that can work with the circuit shown previously in Figure 12.1. You will find the code in the chapter12 folder organized by version:

• chapter12/version1_eventloop is an event-loop-based example.
• chapter12/version2_thread is a thread and callback-based example.
• chapter12/version3_pubsub is a publisher-subscriber-based example.
• chapter12/version4_asyncio is an Asynchronous IO (AsyncIO)-based example.

All versions are functionally equivalent; however, they differ in their code structure and design. We will discuss each version in greater detail after we test our circuit.

Here are the steps to follow to run each version (starting with version 1) and test the circuit:

1. Change to the version1_eventloop folder.
2. Briefly look over the main.py source file, and any additional Python files in the folder, to get a feel for what they contain and how the program is structured.
3. Run...

Exploring the event-loop approach

We will start our code exploration by discussing an event-loop-based approach to building the sample gizmo that we just tested in the previous section.

The code for the event-loop-based approach can be found in the chapter12/version1_eventloop folder. You will find one file named main.py. Please take the time now to stop and read through the code contained in main.py to get a basic understanding of how the program is structured and how it works. Alternatively, you could add breakpoints or insert print() statements into the code and run it again to understand how it works.

How did it go, and what did you notice? If you thought yuck or got lost in the web of loops, if statements, and state variables, then well done! This means you have invested the time to consider this approach and how the code is constructed.

What I mean by an event-loop approach is demonstrated in the code by the while True: loop abbreviated on line 1:

# chapter12/version1_eventloop...

Exploring a threaded approach

Now that we have explored an event-loop-based approach to creating our program, let's consider an alternative approach built using threads, callbacks, and OOP and see how this approach improves code readability and maintainability and promotes code reuse.

The code for the threaded-based approach can be found in the chapter12/version2_threads folder. You will find four files – the main program, main.py, and three class definitions: LED.py, BUTTON.py, and POT.py.

Please take the time now to stop and read through the code contained in main.py to get a basic understanding of how the program is structured and how it works. Then, proceed to review LED.py, BUTTON.py, and POT.py.

How did it go, and what did you notice? I'd guess that you found this version of the program (while reading through main.py) much quicker and easier to understand and noticed that there is no cumbersome and complex while loop, but instead a ...

Exploring the publisher-subscriber alternative

Now that we have seen an approach to creating our program using threads, callbacks, and OOP techniques, let's consider a third approach using a publisher-subscriber model.

The code for the publisher-subscriber approach can be found in the chapter12/version3_pubsub folder. You will find four files – the main program, main.py, and three class definitions: LED.py, BUTTON.py, and POT.py.

Please take the time now to stop and read through the code contained in main.py to get a basic understanding of how the program is structured and how it works. Then, proceed to review LED.py, BUTTON.py, and POT.py.

What you will have noticed is that the overall program structure (especially the class files) is very similar to the version2 thread/callback example that we covered in the previous heading.

You may also have realized that this approach is very similar in concept to the publisher/subscribing method...

Exploring an AsyncIO approach

So far in this chapter, we have seen three different programming approaches to achieving the same end goal. Our fourth and final approach will be built using the AsyncIO libraries offered by Python 3. As we will see, this approach shares similarities and differences with our previous approaches, and also adds an extra dimension to our code and how it operates.

Speaking from my own experience, this approach can feel complex, cumbersome, and confusing the first time you experience asynchronous programming in Python. Yes, there is a steep learning curve to asynchronous programming (and we can only barely scratch the surface in this section). However, as you learn to master the concepts and gain practical experience, you may start to discover that it is an elegant and graceful way to create programs!

If you are new to asynchronous programming in Python, you will find curated tutorial links in the Further reading section to deepen your learning. It is...

An asynchronous experiment

Let's try an experiment. Maybe you've wondered how version4 (AsyncIO) is a bit like our version1 (event-loop) code, only it's been refactored into classes just like the version2 (threaded) code. So, couldn't we just refactor the code in the version1 while loop into classes, create and call a function them (for example, run()) in the while loop, and not bother with all the asynchronous stuff and its extra library and syntax?

Let's try. You will find a version just like this in the chapter12/version5_eventloop2 folder. Try running this version, and see what happens. You'll find that the first LED blinks, the second one is always on, and that the button and potentiometer do not work.

Can you work out why?

Here's the simple answer: in main.py, once the first LED's run() function is called, we're stuck in its while loop forever!

The call to sleep() (from the time library) does not yield control; it just...

Summary

In this chapter, we looked at four different way of structuring a Python program that interface with electronics. We learned about an event-loop approach to programming, two variations on a thread-based approach – callbacks and a publisher-subscriber model – and finished by looking at how an AsyncIO approach to programming works.

Each of the four examples we covered was very discrete and specific in its approach. While we briefly discussed the relative benefits and pitfalls of each approach along the way, it's worth remembering that in practice, your projects will likely use a mixture of these (and potentially other) approaches, depending on the programming and interfacing goals you are trying to achieve.

In the next chapter, we will turn our attention toward IoT platforms and present a discussion of the various options and alternatives that are available for building IoT programs.

Questions

As we conclude, here is a list of questions for you to test your knowledge of this chapter's material. You will find the answers in the Assessments section of the book:

1. When is a publisher-subscriber model a good design approach?
2. What is the Python GIL, and what implication does it present for classic threading?
3. Why is a pure event-loop usually a poor choice for complex applications?
4. Is an event-loop approach a bad idea? Why or why not?
5. What is the purpose of the thread.join() function call?
6. You have used a thread to poll your new analog component via an analog-to-digital converter. However, you find that your code behaves sluggishly to changes in the component. What could be the problem?
7. Which is the superior approach to designing an IoT or electronic interfacing application in Python – using an event-loop, a thread/callback, the publisher-subscriber model, or an AsyncIO-based approach?

Further reading

The realpython.com website has a range of excellent tutorials covering all things concurrency in Python, including the following:

• What is the Python GIL? https://realpython.com/python-gil
• Speed Up Your Python Program with Concurrency: https://realpython.com/python-concurrency
• An Intro to Threading in Python: https://realpython.com/intro-to-python-threading
• Async IO in Python: A Complete Walkthrough: https://realpython.com/async-io-python

The following are relevant links from the official Python (3.7) API documentation:

• Threading: https://docs.python.org/3.7/library/threading.html
• The AsyncIO library: https://docs.python.org/3.7/library/asyncio.htm
• Developing with AsyncIO: https://docs.python.org/3.7/library/asyncio-dev.html
• Concurrency in Python: https://docs.python.org/3.7/library/concurrency.html