Home IoT & Hardware Practical Python Programming for IoT

Practical Python Programming for IoT

By Gary Smart
ai-assist-svg-icon Book + AI Assistant
eBook + AI Assistant $41.99 $28.99
Print $51.99
Subscription $15.99 $10 p/m for three months
ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription.
ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription. $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime! ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription.
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Along with your eBook purchase, enjoy AI Assistant (beta) access in our online reader for a personalized, interactive reading experience.
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription. ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription. BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime! ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription.
eBook + AI Assistant $41.99 $28.99
Print $51.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Along with your eBook purchase, enjoy AI Assistant (beta) access in our online reader for a personalized, interactive reading experience.
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Setting Up your Development Environment
About this book
The age of connected devices is here, be it fitness bands or smart homes. It's now more important than ever to understand how hardware components interact with the internet to collect and analyze user data. The Internet of Things (IoT), combined with the popular open source language Python, can be used to build powerful and intelligent IoT systems with intuitive interfaces. This book consists of three parts, with the first focusing on the "Internet" component of IoT. You'll get to grips with end-to-end IoT app development to control an LED over the internet, before learning how to build RESTful APIs, WebSocket APIs, and MQTT services in Python. The second part delves into the fundamentals behind electronics and GPIO interfacing. As you progress to the last part, you'll focus on the "Things" aspect of IoT, where you will learn how to connect and control a range of electronic sensors and actuators using Python. You'll also explore a variety of topics, such as motor control, ultrasonic sensors, and temperature measurement. Finally, you'll get up to speed with advanced IoT programming techniques in Python, integrate with IoT visualization and automation platforms, and build a comprehensive IoT project. By the end of this book, you'll be well-versed with IoT development and have the knowledge you need to build sophisticated IoT systems using Python.
Publication date:
November 2020
Publisher
Packt
Pages
516
ISBN
9781838982461

 
Setting Up your Development Environment

An important yet often overlooked aspect of Python programming is how to correctly set up and maintain a Python project and its runtime environment. It is often overlooked because it presents as an optional step for the Python ecosystem. And while this might be fine for learning Python language fundamentals, it can quickly become a problem for more complex projects where we need to maintain separate code bases and dependencies to ensure our projects do not interfere with one another, or worse as we will discuss, break operating system tools and utilities.

So, before we jump into IoT code and examples in later chapters, it is so very important for us to cover the steps required to set up a Python project and its run time environment.

In this chapter, we will cover the following topics:

  • Understanding your Python installation
  • Setting up a Python virtual environment
  • Installing Python GPIO packages with pip
  • Alternative methods of executing a Python script
  • Raspberry Pi GPIO interface configuration
 

Technical requirements

To perform the hands-0n exercises in this chapter, you will need the following:

  • Raspberry Pi 4 Model B
  • Raspbian OS Buster (with desktop and recommended software)
  • Minimum Python version 3.5

These requirements are what the code examples in this book are based on. It's reasonable to expect that the code examples should work without modification on a Raspberry Pi 3 Model B or a different version of Raspbian OS as long as your Python version is 3.5 or higher.

The full source code for this book can be found on GitHub at the following URL: https://github.com/PacktPublishing/Practical-Python-Programming-for-IoT. We will clone this repository shortly when we come to the Setting up a Python virtual environment section.

 

Understanding your Python installation

In this section, we will find out which versions of Python you have installed on your Raspberry Pi. As we will discover, there are two versions of Python that come pre-installed on Raspbian OS. Unix-based operating systems (such as Raspbian OS) typically have Python version 2 and 3 pre-installed because there are operating-system-level utilities built with Python.

To find out which versions of Python you have on your Raspberry Pi, follow these steps:

  1. Open a new Terminal and execute the python --version command:

$ python --version
Python 2.7.16

In my example, we see that Python version 2.7.16 has been installed.

  1. Next, run the python3 --version command:

$ python3 --version
Python 3.7.3

In my example, we see that the second version of Python (that is, python3, with the 3) that is installed is version 3.7.3.

Don't worry if the minor versions (the numbers .7.16 after the 2 and .7.3 after 3) are not the same; it is the major versions 2 and 3 that are of interest. Python 2 is a legacy version of Python, while Python 3 is the current and supported version of Python at the time of writing. When we are starting a new Python development, we will practically always use Python 3 unless there are legacy issues we need to contend with.

Python 2 officially became end-of-life in January 2020. It is no longer maintained and will not receive any further enhancements, bug fixes, or security patches.

If you are an experienced Python programmer, you may be able to discern whether a script is written for Python 2 or 3, but it's not always obvious by simply looking at a piece of code. Many new-to-Python developers experience frustrations by mixing up Python programs and code fragments that are meant for different Python versions. Always remember that code written for Python 2 is not guaranteed to be upward-comparable with Python 3 without modification.

A quick tip I can share to visually help to determine which Python version a code fragment is written for (if the programmer has not made it clear in the code comments) is to look for a print statement.

If you look at the following example, you will see that there are two print statements. The first print statement without the parentheses is a give-away that it will only work with Python 2:

print "Hello"  # No parentheses - This only works in Python 2, a dead give-away that this script is for Python 2.

print("Hello") # With parentheses - this will work in Python 2 and Python 3

Of course, you can always run the code against both Python 2 and 3 and see what happens.

We have now seen that there are two Python versions available by default on Raspbian OS, and made mention that there are system-level utilities that are written in Python that reply on these versions. As Python developers, we must take care not to disrupt the global Python installations as this can potentially break system-level utilities.

We will now turn our attention to a very important Python concept, the Python virtual environment, which is the way we isolate or sandbox our own Python projects from the global installation.

 

Setting up a Python virtual environment

In this section, we will discuss how Python interacts with your operating system installation and cover the steps necessary to set up and configure a Python development environment. In addition, as part of our setup process, we will clone the GitHub repository that contains all of the code (organized by chapter) for this book.

By default, Python and its package management tool, pip, operate globally at the system level and can create some confusion for Python beginners because this global default is in contrast to many other language ecosystems that operate locally on a project folder level by default. Unwearyingly working and making changes to the global Python environment can break Python-based system-level tools, and remedying the situation can become a major headache.

As a Python developer, we use Python virtual environments to sandbox our Python projects so they will not adversely interfere with system-level Python utilities or other Python projects.

In this book, we will be using a virtual environment tool known as venv, which comes bundled as a built-in module with Python 3.3 and above. There are other virtual environment tools around, all with their relative strengths and weaknesses, but they all share the common goal of keeping Python dependencies isolated to a project.

virtualenv and pipenv are two alternative virtual environment tool options that offer more features than venv. These alternatives are well suited for complex Python projects and deployments. You'll find links to these in the Further reading section at the end of this chapter.

Let's begin and clone the GitHub repository and create a new Python virtual environment for this chapter's source code. Open a new Terminal window and work through the following steps:

  1. Change into or create a folder where you want to store this book's source code and execute the following commands. With the last command, we rename the cloned folder to be pyiot. This has been done to help shorten Terminal command examples throughout the book:
$ cd ~
$ git clone https://github.com/PacktPublishing/Practical-Python-Programming-for-IoT
$ mv Practical-Python-Programming-for-IoT pyiot

  1. Next, change into the chapter01 folder, which contains the code relating to this chapter:
$ cd ~/pyiot/chapter01
  1. Execute the following command, which creates a new Python virtual environment using the venv tool. It's important that you type python3 (with the 3) and remember that venv is only available with Python 3.3 and above:
$ python3 -m venv venv

The options that we are passing to python3 include -m venv, which tells the Python interpreter that we want to run the module named venvThe venv parameter is the name of the folder where your virtual environment will be created.

While it might look confusing at first glance in the preceding command, it's a common convention to name a virtual environment's folder venv. Later in this chapter, in the Anatomy of a virtual environment section, we will explore what lies beneath the venv folder we just created.
  1. To use a Python virtual environment, we must activate it, which is accomplished with the activate command:
# From with in the folder ~/pyiot/chapter01
$ source venv/bin/activate
(venv) $

When your Terminal has a Python virtual environment activated, all Python-related activity is sandboxed to your virtual environment.

Notice in the preceding code that, after activation, the name of the virtual environment, venv, is shown as part of the Terminal prompt text, that is, (venv) $. In this book, whenever you see Terminal examples where the prompt is (venv) $, it's a reminder that commands need to be executed from within an activated Python virtual environment.
  1. Next, execute which python (without the 3) in your Terminal, and notice that the location of the Python executable is beneath your venv folder and if you check the version of Python, it's Python version 3:
(venv) $ which python
/home/pi/pyiot/chapter01/venv/bin/python

(venv) $ python --version
Python 3.7.3
  1. To leave an activated virtual environment, use the deactivate command as illustrated here:
(venv) $ deactivate
$

Notice also that (venv) $ is no longer part of the Terminal prompt text once the virtual environment has been deactivated.

Remember to type deactivate to leave a virtual environment, not exit. If you type exit in a virtual environment, it will exit the Terminal.
  1. Finally, now that you are outside of our Python virtual environment if you execute which python (without the 3) and python --version again, notice we're back to the default system-level Python interpreter, which is version 2:
$ which python
/usr/bin/python

$ python --version
Python 2.7.13

As we just illustrated in the preceding examples, when we ran python --version in an activated virtual environment, we see that it's Python version 3 whereas in the last example, at the start of this chapter, the system level, python --version, was version 2, and we needed to type python3 --version for version 3. In practice, python (with no number) relates to the default version of Python. Globally, this is version 2. In your virtual environment, we only have one version of Python, which is version 3, so it becomes the default.

A virtual environment created with venv inherits (via a symbolic link) the global Python interpreter version that it was invoked with (in our case, version 3 because the command was python3 -m venv venv). If you ever need to target a specific Python version that is different from the global version, investigate the virtualenv and pipenv virtual environment alternatives.

We have now seen how to create, activate, and deactivate a Python virtual environment and why it is important to use a virtual environment to sandbox Python projects. This sandboxing means we can isolate our own Python projects and their library dependencies from one another, and it prevents us from potentially disrupting the system-level installation of Python and breaking any system-level tools and utilities that rely on them.

Next, we will see how to install and manage Python packages in a virtual environment using pip.

 

Installing Python GPIO packages with pip

In this section, we learn how to install and manage Python packages in a Python virtual environment you created and explored in the previous section. A Python package (or library if you prefer that term) allows us to extend the core Python language with new features and functionality. 

We will need to install many different packages throughout this book, however, for starters and to explore and learn the basic concepts related to package installation and management, we will be installing two common GPIO-related packages in this section that we will use throughout this book. These two packages are the following:

  • The GPIOZero library, an entry-level and easy to use GPIO library for controlling simple electronics
  • The PiGPIO library, an advanced GPIO library with many features for more complex electronic interfacing

In the Python ecosystem, package management is done with the pip command (pip stands for Python installs packages). The official public package repository that pip queries is known as the Python Package Index, or simply PyPi, and it is available for browsing on the web at https://pypi.org.

Similarly to python and python3, there is pip and pip3. pip (without the number) will be the default pip command that is matched to the default python command in a given virtual environment.

There will be code examples in this book where we will be interacting with your Raspberry Pi's GPIO pins, so we need to install a Python package (or two) so that your Python code can work with your Raspberry Pi's GPIO pins. For now, we are just going to check for and install two GPIO-related packages. In Chapter 2Getting Started with Python and IoT, and Chapter 5, Connecting Your Raspberry Pi to the Physical World, we will cover these GPIO packages and other alternatives in greater detail.

In your chapter01 source code folder, you will find a file named gpio_pkg_check.py, which is replicated in the following. We will use this file as the basis to learn about pip and package management in the context of a Python virtual environment. This script simply reports the availability of a Python package depending on whether using import succeeds or raises an exception:

"""
Source File: chapter01/gpio_pkg_check.py
"""
try:
import gpiozero
print('GPIOZero Available')
except:
print('GPIOZero Unavailable. Install with "pip install gpiozero"')

try:
import pigpio
print('pigpio Available')
except:
print('pigpio Unavailable. Install with "pip install pigpio"')

Let's check for the availability of GPIO packages using gpio_pkg_check.py and with pip. I'll kill the suspense by telling you that they're not going to be available in your freshly-created virtual environment (yet), however, we are going to install them!

Note: They are already installed at the system level if you want to check yourself by running this script outside of your virtual environment.

The following steps will walk us through the process of upgrading pip, exploring the tool's options, and installing packages:

  1. As the first step, we will upgrade the pip tool. In a Terminal window, run the following command, remembering that all commands that follow must be performed in an activated virtual environment—meaning you should see the text (venv) in the Terminal prompt:
(venv) $ pip install --upgrade pip
...output truncated...

The preceding upgrade command may take a minute or two complete and will potentially output a lot of text to the Terminal.

Are you facing pip problems? If you're getting a sea of red errors and exceptions when trying to install a package with pip, try upgrading the pip version as a first step using pip install --upgrade pip. It is a recommended first step after creating a fresh Python virtual environment to upgrade pip.
  1. With pip now upgraded, we can see what Python packages are already installed in our virtual environment using the pip list command:
(venv) $ pip list
pip (9.0.1)
pkg-resources (0.0.0)
setuptools (33.1.1)

What we see in the preceding are the default Python packages in our fresh virtual environment. Do not worry if the exact package list or version numbers do not match exactly with the example.

  1. Run our Python script with the python gpio_pkg_check.py command and observe that our GPIO packages are not installed:
(venv) $ python gpio_pkg_check.py
GPIOZero Unavailable. Install with "pip install gpiozero"
pigpio Unavailable. Install with "pip install pigpio"
  1. To install our two required GPIO packages, we use the pip install command as shown in the following example:
(venv) $ pip install gpiozero pigpio
Collecting gpiozero...
... output truncated ...
  1. Now, run the pip list command again; we will see these new packages are now installed in our virtual environment:
(venv) $ pip list
colorzero (1.1)
gpiozero (1.5.0) # GPIOZero
pigpio (1.42) # PiGPIO
pip (9.0.1)
pkg-resources (0.0.0)
setuptools (33.1.1)

You may have noticed that there is a package called colorzero (this is a color manipulation library) that we did not install. gpiozero (version 1.5.0) has a dependency on colorzero, so pip has installed it for us automatically.

  1. Re-run python gpio_pkg_check.py and we now see that our Python modules are available for import:
(venv) $ python gpio_pkg_check.py
GPIOZero Available
pigpio Available

Great! We now have a virtual environment with two GPIO packages installed. As you work on Python projects, you will inevitably install more and more packages and want to keep track of them.

  1. Take a snapshot of the packages you have previously installed with the pip freeze command:
(venv) $ pip freeze > requirements.txt

The preceding example freezes all installed packages into a file named requirements.txt, which is a common filename to use for this purpose.

  1. Look inside the requirements.txt file and you will see all of the Python packages listed together with their version numbers:
(venv) $ cat requirements.txt
colorzero==1.1
gpiozero==1.5.0
pigpio==1.42
pkg-resources==0.0.0

In the future, if you move your Python project to another machine or a new virtual environment, you can use your requirement.txt file to install all of your captured packages in one go using the pip install -r requirements.txt command.

Our requirements.txt example shows we have installed GPIOZero version 1.5.0, the current version at the time of writing. This version has a dependency on ColorZero version 1.1. It is possible that different (past or future) versions of GPIOZero may have different dependencies than those shown in our example, so your own requirements.txt file when performing the example exercise may be different.

We've now completed the basic installation life cycle of Python packages using pip. Note that whenever you install new packages with pip install, you also need to re-run pip freeze > requirements.txt to capture the new packages and their dependencies.

To finish our exploration of pip and package management, here are a few other common pip commands:

# Remove a package
(venv) $ pip uninstall <package name>

# Search PyPi for a package (or point your web browser at https://pypi.org)
(venv) $ pip search <query text>

# See all pip commands and options (also see Further Reading at the end of the chapter).
(venv) $ pip --help

Congratulations! We've reached a milestone and covered the essential virtual environment principles that you can use for any Python project, even ones that are not Raspberry Pi related! 

During your Python journey, you will also come across other package installers and tools named easy_install and setuptools. Both have their uses; however, it's pip that you will rely on most of the time.

Now that we have seen how to create a virtual environment and install packages, let's take a look at a typical Python project folder structure such as ~/pyiot/chapter01 and discover what lies beneath the venv folder.

 

Anatomy of a virtual environment

This section relates to venv, which we have been using in this chapter, and will apply to virtualenv but not pipenvwhich we listed as alternative virtual environment tools. The example is also specific to a Raspbian OS and is typical of a standard Unix-based OS. It's important to, at a minimum, understand the basic structure of a virtual environment deployment since we will be mixing our own Python programming code in with the files and folders that make up the virtual environment.

The light weight venv tool that comes with Python 3.3 and above is a subset of virtualenv.

Here is the folder structure of our virtual environment. Yep, its a screenshot from a Mac. That's so I could get everything on screen at once:

Figure 1.1 – Contents of a typical venv virtual environment folder

The following points explain the core subfolders that are found within our ~/pyiot/chapter01 folder after we ran python3 -m venv venv and installed packages using pip:

  • The venv folder contains all of the Python virtual environment files. There is no real practical need to be touching anything under this folder manually—let the tools do that for you. Remember that the folder is named venv only because that's what we called it when it was created.
  • The venv/bin folder contains the Python interpreter (in the venv case, there are symbolic links to the system interpreter) and other core Python tools, including pip.
  • Underneath the venv/lib folder are all the sandboxed Python packages for the virtual environment, including the GPIOZero and PiGPIO packages we installed using pip install.
  • Our Python source file, gpio_pkg_check.pyis in the top-level folder, ~/pyiot/chapter01; however, you can create sub-folders here to help to organize your code and non-code files.
  • Finally, requirements.txt lives by convention in the top project folder.

The virtual environment folder venv does not actually need to be kept in the project folder; however, it's often convenient to have it there for activation with the activate command.

Your venv folder and anything below it should not be added to your source version control system, but you should add requirements.txtAs long as you have a current requirements.txt file, you can always recreate your virtual environment and reinstate packages to a known state.

It's important to understand that, as a Python developer, you will be mixing in your own programming code with files and folders that form part of the virtual environment system and that you should be pragmatic when selecting which files and folders are added to your version control system, should you be using one.

This last point is important since the virtual environment system can amount to many megabytes in size (and often many times larger than your program code) that does not need versioning (since we can always recreate the virtual environment as long as we have a requirements.txt file), plus it's host platform-specific (that is, there will be differences between Windows, Mac, and Linux), plus there will be differences between different virtual environment tools (for example, venv versus pipenv). As such, virtual environments are not generally portable in projects that involve many developers working on different computers.

Now that we have briefly explored the file and folders structure and the importance of understanding this structure, we will continue and look at alternative ways of running a script that is sandboxed to a virtual environment.

 

Alternative methods of executing a Python script

Let's briefly turn our attention to the alternative ways that we can execute a Python script. As we will learn, choosing the appropriate method is all based around how and from where you intend to start your script and whether your code requires elevated permissions.

The most common way of running a Python script is from within its virtual environment and with the permissions of the currently logged in user. However, there will be scenarios where we need to run a script as the root user or from outside an activated virtual environment.

Here are the ways we will explore:

  • Using sudo with virtual environments
  • Executing Python scripts outside of their virtual environments
  • Running a Python script at boot

Let's start by learning how to run a Python script with root user permissions.

 

Using sudo within virtual environments

I'm sure that while working on your Raspberry Pi you have had to execute commands in a Terminal with the sudo prefix because they required root privileges. If you ever need to run a Python script that is in a virtual environment as root, you must use the full path to your virtual environment's Python interpreter.

Simply prefixing sudo before python, as shown in the following example, does not work under most circumstances, even if we are in the virtual environment. The sudo action will use the default Python that's available to the root user, as shown in the second half of the example:

# Won't work as you might expect!
(venv) $ sudo python my_script.py

# Here is what the root user uses as 'python' (which is actually Python version 2).
(venv) $ sudo which python
/usr/bin/python

The correct way to run a script as root is to pass the absolute path to your virtual environment's Python interpreter. We can find the absolute path using the which python command from inside an activated virtual environment:

(venv) $ which python
/home/pi/pyiot/chapter01/venv/bin/python

Now, we sudo our virtual environment's Python interpreter and the script will run as the root user and within the content of our virtual environment:

(venv) $ sudo /home/pi/pyiot/chapter01/venv/bin/python my_script.py

Next, we'll see how to run a Python script that's sandboxed in a virtual environment from outside of its virtual environment. 

 

Executing Python scripts outside of their virtual environments

A natural extension to the preceding discussion on sudo is how do I run a Python script from outside of its virtual environment? The answer is the same as in the preceding section: just make sure you are using the absolute path to your virtual environment's Python interpreter.

Note: In the following two examples, we're not in a virtual environment—there is no $ (venv) on the prompt. If you still need to exit your Python virtual environment, type deactivate.

The following command will run a script as the currently logged in user (which, by default, is the pi user):

# Run script as logged-in user.
$ /home/pi/pyiot/chapter01/venv/bin/python gpio_pkg_check.py

Or to run the script as root, prefix sudo:

# Run script as root user by prefixing sudo
$ sudo /home/pi/pyiot/chapter01/venv/bin/python gpio_pkg_check.py

Since we are using the virtual environment's Python interpreter, we are still sandboxed to our virtual environment and any Python packages we installed are available.

Next, we will learn how to make a Python script run whenever you boot your Raspberry Pi.

 

Running a Python script at boot

There will come a time when you have developed an awesome IoT project and you want it to run automatically every time you start your Raspberry Pi. Here is one simple way to achieve this using a feature of cron, the Unix scheduler. If you are not familiar with the basics of cron, search the web for cron tutorial—you'll find heaps of them. I've provided curated links in the Further reading section.

Here are the steps to configure cron and make a script run on boot:

  1. In your project folder, create a bash script. I've named it run_on_boot.sh:
#!/bin/bash

# Absolute path to virtual environment python interpreter
PYTHON=/home/pi/pyiot/chapter01/venv/bin/python

# Absolute path to Python script
SCRIPT=/home/pi/pyiot/chapter01/gpio_pkg_check.py

# Absolute path to output log file
LOG=/home/pi/pyiot/chapter01/gpio_pkg_check.log

echo -e "\n####### STARTUP $(date) ######\n" >> $LOG
$PYTHON $SCRIPT >> $LOG 2>&1

This bash script will run a Python script using the absolute paths for both the script and its Python interpreter. Also, it captures any script output and stores it in a log file. For this example, we're simply going to run and log the output of gpio_pkg_check.py on boot. It's the last line that ties everything together and runs and logs our Python script. The 2>&1 part at the end is necessary to ensure that errors, in addition to standard output, are also logged.

  1. Mark the run_on_boot.sh file as an executable file:
$ chmod u+x run_on_boot.sh

If you are not familiar with the chmod command (chmod means change mode), what we are doing is giving the operating system permission to execute the run_on_boot.sh file. The u+x parameters mean for the current User, make the file eXecutable. To learn more about chmod, you can type chmod --help or man chmod in the Terminal.

  1. Edit your crontab file, which is the file where cron scheduling rules are stored:
$ crontab -e
  1. Add the following entry to your crontab file, using the absolute path to the run_on_boot.sh bash script we created in step 1:
@reboot /home/pi/pyiot/chapter01/run_on_boot.sh &

Do not forget the & character at the end of the line. This makes sure the script runs in the background.

  1. Run the run_on_boot.sh file manually in a Terminal to make sure it works. The gpio_pkg_check.log file should be created and contains the output of the Python script:
$ ./run_on_boot.sh
$ cat gpio_pkg_check.log
####### STARTUP Fri 13 Sep 2019 03:59:58 PM AEST ######
GPIOZero Available
PiGPIO Available
  1. Reboot your Raspberry Pi:
$ sudo reboot
  1. Once your Raspberry Pi has finished restarting, the gpio_pkg_check.log file should now contain additional lines, indicating that the script did indeed run at boot:
$ cd ~/pyiot/chapter01
$ cat gpio_pkg_check.log

####### STARTUP Fri 13 Sep 2019 03:59:58 PM AEST ######

GPIOZero Available
PiGPIO Available

####### STARTUP Fri 13 Sep 2019 04:06:12 PM AEST ######

GPIOZero Available
PiGPIO Available

If you are not seeing the additional output in the gpio_pkg_check.log file after a reboot, double-check that the absolute path you entered in crontab is correct and that it works manually as per step 5. Also, review the system log file,  /var/log/syslog, and search for the text, run_on_boot.sh.

Our cron-based example of running a script on boot is one of many options that are available in Unix-based operating systems such as Raspbian. Another common and more advanced option using systemd can be found on the Raspberry Pi website at https://www.raspberrypi.org/documentation/linux/usage/systemd.md. Irrespective of the option you prefer, the key point to remember is to ensure your Python scripts run from within their virtual environment.

We have now learned alternative methods to run a Python script, which will help you in the future to correctly run your Python-based IoT projects after they are developed or start them when your Raspberry Pi boots if required.

Next, we will now move on to making sure your Raspberry Pi is set up and configured correctly for the GPIO and electronic interfacing that we'll be diving into in the next chapter, Chapter 2, Getting Started with Python and IoT, and subsequent chapters.

 

Configuring the GPIO interface on our Raspberry Pi

Before we can start working with Python GPIO libraries and controlling electronics, one task we need to perform is to enable the GPIO interfaces on your Raspberry Pi. Even though we have installed Python packages for GPIO control, we have not told Raspbian OS that we want to use the Raspberry Pi's GPIO Pins for specific cases. Let's do that now.

Here are the steps to follow:

  1. From your Raspbian desktop, navigate to the Raspberry menu | PreferencesRaspberry Pi Configuration, as shown here in Figure 1.2:

Figure 1.2 Location of the Raspberry Pi Configuration menu item
Alternatively, interfaces can be managed at the command line with the sudo raspi-config command and navigating to the Interfacing Options menu.
  1. Enable all of the interfaces as shown in the following screenshot:

Figure 1.3 - Raspberry Pi Configuration Dialog
  1. Click the OK button.

After you click the OK button, you may be prompted to reboot your Raspberry Pi; however, do not confirm the reboot just yet because there is one more task we need to perform first. We'll look at that next.

 

Configuring the PiGPIO daemon

We also need to start the PiGPIO daemon, which is a system service that needs to be running so that we can use the PiGPIO GPIO client library, which we will start using next in Chapter 2, Getting Started with Python and IoT.

Architecturally, the PiGPIO library comprises two partsa server service and a client that communicates over local pipes or sockets to the service. We will cover more about this basic architecture in Chapter 5Connecting Your Raspberry Pi to the Physical World.

Execute the following in a Terminal. This will start the PiGPIO daemon and will ensure that the PiGPIO daemon is started automatically when your Raspberry Pi boots:

$ sudo systemctl enable pigpiod
$ sudo systemctl start pigpiod

Now, it's time to reboot your Raspberry Pi! So, take a break while your Raspberry Pi restarts. You deserve it because we have covered a lot! 

 

Summary

In this chapter, we explored the Python ecosystem that is part of a typical Unix-based operating system such as Raspbian OS and learned that Python is a core element of the operating system tooling. We then covered how to create and navigate a Python virtual environment so that we can sandbox our Python projects so they will not interfere with one another or the system-level Python ecosystem.

Next, we learned how to use the Python package management tool, pip, to install and manage Python library dependencies from within a virtual environment, and we did this by installing the GPIOZero and PiGPIO libraries. And since there will be times that we need to execute a Python script as the root user, from outside its virtual environment or during boot up, we also covered these various techniques.

By default, Raspbian does not have all of its GPIO interfaces enabled, so we performed the configuration needed to enable these features so that they are readily available for use in later chapters. We also started and learned how to set up the PiGPIO daemon service so that it starts every time your Raspberry Pi is booted.

The core knowledge you have gained in this chapter will help you to correctly set up and navigate sandboxed Python development environments for your own IoT (and non-IoT) projects and safely install library dependencies so they do not interfere with your other Python projects or the system-level installation of Python. Your understanding of different ways of executing a Python program will also help you to run your projects with elevated user permissions (that is, as the root user), or at boot, should your project have these requirements.

Next, in Chapter 2Getting Started with Python and IoT, we will jump straight into Python and electronics and create an end-to-end internet-enabled program that can control an LED over the internet. We will take a look at two alternative ways of flashing an LED using the GPIOZero and PiGPIO GPIO libraries before connecting our LED to the internet by using an online service, dweet.io, as our networking layer. 

 

Questions

As we conclude, here is a list of questions for you to test your knowledge regarding this chapter's material. You will find the answers in the Assessments section of the book:

  1. What is the main reason why you should always use a virtual environment for your Python projects?
  2. Do you need to or should you place the virtual environment folder (that is, venv) under version control?
  3. Why create a requirements.txt file?
  4. You need to run a Python script as the root user. What step must you take to ensure that the script executes in its intended virtual environment context?
  5. What does the source venv/bin/activate command do?
  6. You are in an activated virtual environment. What is the command to leave the virtual environment and return to the host shell?
  7. You created a Python project and virtual environment in PyCharm. Can you work on and run the project's Python scripts in a Terminal?
  8. You want a GUI tool to edit and test Python code on your Raspberry Pi but do not have PyCharm installed. What pre-installed tool that comes with Python and Raspbian could you use?
  9. You've advanced in your Python and electronics knowledge and are trying to hook up a device using I2C to your Raspberry Pi but you cannot get it to work. What might be the problem and how do you address it?
 

Further reading

We covered the venv virtual environment tool in this chapter. Here are links to its official documentation:

If you would like to learn about the virtualenv and pipenv alternative virtual environment tools, here is their official documentation:

The following is a link to the Python Packaging Guide. Here you will find a comprehensive guide regarding Python package management, including pip and the easy-install/setup tools alternatives: 

If you wish to learn more about scheduling and cron, here are two resources to get you started:

Latest Reviews (5 reviews total)
Good product. Just Started reading.
DON'T HAVE MORE COMMENTS!!!
Practical Python Programming for IoT
Unlock this book and the full library FREE for 7 days
Start now