In recent years, the Internet of Things (IoT) has been a trending field for research and development. The future of IoT has the potential to be limitless. By 2025, it is estimated that there will be more than 21 billion IoT devices. So, wouldn’t it be great if you could add these to your known technological stacks? In this book, we will start with the absolute basics.
I will walk you through two fascinating subjects throughout this book: Raspberry Pi, which is a prevalent development board for beginners, and MQTT, a very commonly used and robust communication protocol to delve into the world of IoT.
This chapter will introduce you to the basics of MQTT and Raspberry Pi. Moreover, it will help you set up your Raspberry Pi. Although simple, it is crucial to perform each step as this will help us set up our own local MQTT broker on the Raspberry Pi. This will also help you understand how to get started with your new Raspberry Pi by installing an operating system onto it.
First, we will flash the popular Debian-based Raspberry Pi OS on our Raspberry Pi. After that, we will install all the necessary libraries and packages to make our device a local MQTT broker.
We will cover the following main topics in this chapter:
So, let’s start by knowing what MQTT is.
In this section, we will learn about the essential concepts of MQTT. First, we will look at the basic concepts of MQTT and some history, followed by the functionality and components of MQTT. Finally, we will have a brief encounter with the salient features of MQTT.
Please note that there are different versions of MQTT, and most of what we discuss is relevant to the MQTT protocol version 3.1.
MQTT stands for Message Queuing Telemetry Transport. It is a lightweight communication protocol.
According to the official MQTT v3.1 documentation:
MQTT is a Client-Server publish/subscribe messaging transport protocol. It is lightweight, open, simple, and designed to be easy to implement. These characteristics make it ideal for use in many situations, including constrained environments for communication in Machine to Machine (M2M) and Internet of Things (IoT) contexts where a small code footprint is required, and network bandwidth is at a premium.
This is a clear and clean definition of the MQTT protocol in just a few lines. It is a messaging protocol designed for easy implementation, primarily client side. It is an open and lightweight communication protocol with minimal packet overhead. It is generally used for communication between two or more devices.
Now that we know what MQTT is, we will explore the basic concepts of MQTT we came across in the previous section. More specifically, we will look at MQTT as a publish/subscribe protocol.
The publish/subscribe protocol is an alternative to the traditional client-server architecture. It means that instead of categorizing both sending and receiving machines as clients, the clients who send a message are publishers and the clients who receive the messages are the subscribers.
Another essential feature of such protocols is the decoupling between the clients. In simple words, the clients never directly communicate with each other. They are mediated by the third component of this system, known as the broker. In this book, we will be using our Raspberry Pi as an MQTT broker, which connects different client devices within a local network. The primary function of the broker is to mediate and manage all communications between the various clients (i.e., publishers and subscribers).
To better understand how the whole system works, please see Figure 1.1, which shows how the communication protocol operates with a very simplified diagram. In this example, the publishing client is a temperature monitor and the subscribing device is a mobile phone:
Figure 1.1 – Basic MQTT communication flow
Please note that this is just a simplified representation. There can be multiple publishers and subscribers connected to a single broker. As you can see, the temperature monitor sends the current temperature value of 27°C through the MQTT communication protocol, which is then received by the MQTT broker, which routes it to the subscriber, a mobile application in our case.
We will now look into some details about publishers, subscribers, and brokers with the help of an example:
Please note that both the publishers and subscribers are referred to as clients. A client can be a publisher, subscriber, or both as both these processes are entirely independent of each other, as we will see in the later chapters.
But another question arises now: how does the broker manage or route which information is sent where? The following section will answer this question, exploring MQTT functionality.
We have already seen the significant components of MQTT, but we will now explore how these components communicate with each other.
MQTT has no client device addresses or identifiers, making it easy to build an expansible, ad hoc network. The only thing all clients must know is the address of the broker. So, how do messages get routed between the clients? The solution for this is topics and messages.
This is how the whole system works:
Kitchen/Fridge/
The main topic is the kitchen, and the appliance is the subtopic. The message will be Temperature:14
on the given topic.
Kitchen
topic, it will have access to all the subtopics that are a part of this topic.Now that we are aware of the details of MQTT, we will have a look at the salient features of this communication protocol.
This section will cover the main features of this communication protocol:
MQTT clients are tiny, and they require minimal resources to operate. So, even microcontrollers such as ESP8266 can be used as a client as long as they have an active connection to a network.
This protocol is highly efficient thanks to the small message headers that provide maximum network bandwidth efficiency.
MQTT allows to-and-fro messaging capability. This means a device can be a publisher and a subscriber simultaneously. This also allows easy broadcasting of messages to several devices at once.
There is no worry about maintaining clients’ addresses or IDs; it is effortless to expand the MQTT network. Moreover, the decoupling between the publishers and subscribers makes things even more accessible. The only things required on the client side are the broker’s IP address and the topic name.
MQTT is highly reliable when it comes to message delivery. As this is an essential aspect of any communication protocol, MQTT comes with three predefined quality of service (QoS) levels:
Many IoT devices are connected over unreliable networks, and MQTT’s support for persistent sessions reduces the client’s time with the broker. For example, several monitoring devices are deployed on moving vehicles or in remote areas such as forests.
MQTT makes it easy to encrypt messages using TLS and authenticate clients using modern authentication protocols, such as OAuth.
This is the end of this section. We covered the basics of MQTT and the components and salient features of this popular communication protocol. This protocol will be discussed in detail in Chapter 2,MQTT in Detail, of this book.
This section will introduce you to the Raspberry Pi, the world’s most affordable credit card-sized computer.
There is a wide range of available Raspberry Pi development boards available. They are primarily available in four formats:
The latest models of the Pi available are as follows:
We will cover the Raspberry Pi Model 4B in depth as it is the latest variant available and is the model we will be using throughout the book.
This is the latest development board from Raspberry Pi (Figure 1.2). It has several new and improved features that make it an incredible upgrade over the older models. The most significant change is the support of two 4K displays, which is an astonishing feat on hardware that costs 35 dollars.
Another distinctive feature is the upgraded CPU and RAM options. The latest board is powered by a new 1.5 GHz quad-core CPU, almost three times faster than the previous-generation processor. Moreover, the boards are available in 2 GB, 4 GB, and 8 GB LPDDR4 RAM configurations.
It also has USB C support, USB 3.0 support, and Gigabit Ethernet. The Raspberry Pi 4 is a viable dual-display desktop computer with these new hardware capabilities.
Figure 1.2 – Raspberry Pi Model 4B+: the latest Raspberry Pi development board
Now, we will dig a bit deeper and cover the hardware specifications of this development board in detail followed by a brief discussion of some popular operating systems that are available for this board.
The hardware specifications of the Raspberry Pi model 4 are as follows, as mentioned on the official Raspberry Pi 4 product page:
The following figure shows the available ports and some technical specifications of the Raspberry Pi 4:
Figure 1.3 – Raspberry Pi 4 ports and hardware specifications
Now that we are done with the hardware specifications, let’s move toward the available software options. As the Pi is a full-blown computer, it will run an operating system of its own. Hence, we will look at some popular operating systems available for the Raspberry Pi.
There are several operating systems available for the Raspberry Pi. We will look at some of the most popular operating systems listed on their official website:
Now that we have some knowledge about Raspberry Pi and MQTT, the next step is to learn how to setup our Raspberry Pi so that we can use it as a MQTT broker. That is exactly what the next section is about!
In this section, we will cover how to set up the Raspberry Pi. Although simple, it is crucial to perform each step as this will help us set up our own local MQTT broker on the Raspberry Pi. This will also help you understand how to get started with your new Raspberry Pi by installing an operating system onto it.
First, we will flash the popular Debian-based Raspberry Pi OS on our Raspberry Pi. After that, we will install all the necessary libraries and packages to make our device a local MQTT broker.
The topics we will be covering will be as follows:
First, we will discuss what will be required in terms of hardware to follow this setup process.
To follow the instructions provided in this section, you will need the following hardware:
So, let us proceed to the next step, which is installing the official Raspberry Pi OS image and setting up the SD card.
In this step, the main aim is to get the microSD Card ready for the Raspberry Pi.
Important Note
If you have purchased a Raspberry Pi bundle with a pre-burnt SD card and some optional accessories, you can skip this step, as the SD card you have is already loaded with the required OS. But, if you would like to install a different OS, you can follow this step to do so.
Before installing the OS onto the SD card, we need to format the SD card to make sure nothing corrupts the OS. There are two methods to do so. The first is the easy way, using software to do this task, and this can be done in Windows and macOS systems. The second method is a little more complicated, and it will cover how to do the same for Linux-based systems.
The SD Card Formatter software (managed by the SD Association) helps you wipe the SD card totally so it can be used for the desired purpose, which, in our case, is to burn an OS image onto it.
We need to perform this step when the SD card we are using has been previously used or has some data stored on it already. This can corrupt the OS, and so all the existing data needs to be wiped. Moreover, this is perfect practice and should be done every time we install a new OS.
Let’s look at the following steps:
Figure 1.4 – Download page for SD Card Formatter
Figure 1.5 – The dialog box that appears after successful installation of the software
Figure 1.6 – Common SD card adapters
Figure 1.7 – SD Card Formatter application page
Figure 1.8 – Formatting completion dialog box
We have successfully formatted our SD card! Now, we are all set to burn our Raspberry Pi OS image onto our card.
Important Note
Please keep in mind that if you are using an SD card adapter, make sure it is in unlock mode or you could face formatting issues. Unlocking the adapter means giving the computer access to the SD card. This is done by simply flipping a small switch on the side of the adapter.
We will use GParted to format our SD card on a Linux system. It is an open source disk management software. Just follow these steps:
sudo apt install gparted
Once the app has been installed, it will be available in the Applications menu. Just find and launch the application, as shown in the following figure:
Figure 1.9 – Launching GParted from the Applications menu
Figure 1.10 – GParted application home screen
The user interface is relatively easy to navigate and much more intuitive than the one we get with the preinstalled disk utility software. There are additional features that this software provides, such as creating bootable USB drives and downloading an ISO file, for example.
Figure 1.11 – Selecting the USB drive to format
Figure 1.12 – Unmounting the USB adapter before formatting
Figure 1.13 – Selecting the desired file format (fat32 in most cases)
This won’t start the formatting process. It will just add a new operation to the list of pending operations, as seen here:
Figure 1.14 – List of pending operations
Figure 1.15 – Clicking on Apply All Operations
Figure 1.16 – Clicking Apply to continue formatting
The formatting process will start. You can track the progress using the window that pops up:
Figure 1.17 – Formatting progress dialog box – GParted
You will see the window shown in the following screenshot once the formatting process has been completed:
Figure 1.18 – Formatting process completed dialog box
Your SD card is formatted! Close the application, and your USB adapter will be listed in the file manager. We can now move to the next step.
Now that our SD card is ready for the Raspberry Pi OS, the next step is to install the Raspberry Pi Imager software, which is the easiest way to install the OS onto our SD card.
Important Note
For more advanced users who are looking to install a particular OS, follow this link: https://www.raspberrypi.org/documentation/installation/installing-images/README.md.
In the next section, we’re going to follow the step-by-step process to install the Raspberry Pi OS on your SD card.
The first step is to install the software:
Figure 1.19 – Downloading Raspberry Pi Imager
Figure 1.20 – Raspberry Pi Imager application
Figure 1.21 – All the available OS options
Figure 1.22 – Choosing the SD card you want to burn to OS to
Figure 1.23 – Pressing the Write button
This will start the writing process. The application will first download the image file you chose and then write it onto the SD card. Please note that you will not see the download progress, just a Writing progress bar. So, it is an excellent time to grab a cup of coffee or go for a short walk, as this may take some time:
Figure 1.24 – You can see the writing progress in the application
This completes the SD card preparation for our Raspberry Pi. In the next step, we will boot into our new OS for the first time and update and upgrade some software to the latest version, enabling VNC to wirelessly SSH into our Pi (don’t worry, we will discuss this in detail in a later section).
After the OS has been written to the SD card, we will insert this card into our Raspberry Pi, as seen in the following figure:
Figure 1.25 – Inserting the SD card into the Pi (image from the official Raspberry Pi website)
We will also connect a display using either a display port (if you have a Raspberry Pi 4) or a simple HDMI display, and a USB keyboard and mouse. Finally, after all the peripherals are connected, we will connect our power supply (any USB C or micro-USB charger, depending on the model you are using) to the Pi. Please see the following figure for how to make the connections:
Figure 1.26 – Powering the Raspberry Pi after connecting the peripherals
Once all the setup is complete, connect the power supply to the Pi, and you should see the Pi booting up on the connected HDMI screen. It will take 20-30 seconds for the first boot. Once it is done, a welcome screen will appear saying Welcome to Raspberry Pi Desktop, as seen in Figure 1.27:
Figure 1.27 – Raspberry Pi welcome screen
Just complete the first-time setup by clicking on the Next button. It will first ask you to set the location settings and choose the language and keyboard accordingly:
Figure 1.28 – Setting up the location, language, and timezone
After the required information has been entered, press the Next button. It will take a few seconds for the system to set up the location.
Next, the OS will prompt you to change the default password of your system, which is raspberry. Select a strong password, and after entering all the required information, press the Next button:
Figure 1.29 – Setting up a new password
After this, the system will help you choose the best resolution according to your display type. You can skip this step, as we will only use this monitor or display once. In the next section, we will learn how to set up VNC and SSH on our Pi to access it wirelessly when we are connected to the same network.
Next, we need to connect to a Wi-Fi network, as shown in Figure 1.30. Select your network from the list of available networks, then authenticate by entering your password to connect to your network. If you have connected via Ethernet or would like to do it later, you can skip this step by pressing the Skip button.
Please note that you will require a shared Wi-Fi network to use VNC, and you will need a local network at the very least. If you skip the step, for now, you can always connect to a network through the Wi-Fi symbol on the top-right side of the desktop in the following figure:
Figure 1.30 – Connecting to a Wi-Fi network
Finally, the last step of the setup is to update the software to the latest version. As we have used the newest version of Raspberry Pi OS, no significant updates will be pending. It is still preferable to use this opportunity to update all the preinstalled software to the latest version (Figure 1.31).
Just click Next as given in the instructions, and the process will automatically start:
Figure 1.31 – Updating to the latest software
Important Note
If you fail to update your system through this setup, it is possible to do this later. For this, open the terminal (the black icon on the top-left side of the home screen) and type the following commands:
sudo apt update
– This will fetch the list of all available updates.
sudo apt dist-upgrade
– This will download and install the updates.
It will take some time to fetch all the updates and install them, so this would be an excellent time to go for a short walk.
After the updates have been successfully downloaded and installed, the system will prompt you to restart the system for all the changes to take effect. You can do so by simply pressing the Restart button, as shown in the following figure:
Figure 1.32 – Restarting the system after the first setup
This completes the first-time setup of our Raspberry Pi! In the next section, we will learn how to set up VNC on our Raspberry Pi to access it wirelessly through our PC. What’s more, we can also use our PC or laptop keyboard and mouse with the Pi.
In this step, we will learn how to set up SSH (Secure Shell) and VNC (Virtual Network Computing) on the Raspberry Pi. Note that the completion of the previous steps of the OS installation and setup on the Raspberry Pi is required to set these up.
Before getting into the practical part, let’s learn a bit more about VNC and SSH:
Figure 1.33 – How does SSH work?
We will be using the RealVNC Server software, which is preinstalled on our Pi. Raspberry Pi will act as a VNC server, and a client software would be needed to access it through the main computer.
Let’s get started on the steps to set up VNC:
Preferences | Raspberry Pi Configuration
Figure 1.34 – Selecting Raspberry Pi Configuration
Figure 1.35 – Raspberry Pi Configuration Interfaces
Figure 1.36 – Enabling SSH and VNC
This will enable both the protocols on your Pi. Now, the Raspberry Pi is remotely accessible. Congratulations! To verify that it is working, check the top-left part of the screen. A white-colored VNC logo should be visible now:
Figure 1.37 – Checking for the VNC logo when enabled
Figure 1.38 – Make a note of the IP address
This is what the Download page looks like:
Figure 1.39 – The VNC Viewer download page
Figure 1.40 – The VNC Viewer application
Figure 1.41 – The Authentication window
Figure 1.42 – Remote access to Raspberry Pi
You now have access to your Raspberry Pi wirelessly with the ability to use your computer mouse and keyboard with your Pi’s OS.
The next step is crucial, as we will install a software library that will allow us to use our Raspberry Pi as an MQTT broker. We will install the following two libraries developed by Mosquitto:
After this setup step and the testing step that follows, you will be able to host a local MQTT broker on your Raspberry Pi device. Moreover, you will also test whether your broker is successfully running by simultaneously running a client code on the Pi itself. Cool, right?
So, let’s get started with the steps to get this done.
Mosquitto is a popular MQTT broker well-supported on Debian-based Linux platforms such as Raspbian. To install this package, just open a new terminal, then type in the following command. It’s easy to install using the apt
package installer:
sudo apt install mosquitto mosquitto-clients
This command requires root privileges for which we have used sudo
in our command. After that, the installation process will start, and once it is complete, you should see an output as shown in the following screenshot:
Figure 1.43 – Terminal view while installing the packages
The mosquitto-clients
package is optional for running the Pi as an MQTT broker, but it will help us test whether the broker is running locally.
This package allows you to use your Raspberry Pi as an MQTT client as well. So, if you want to create a local dashboard to control all the MQTT clients from your Pi, you will be able to do so.
The broker is still not active. To enable it, type in the following command in your terminal window:
sudo systemctl enable mosquitto
systemd
is a Linux package manager that will help you monitor and control the different applications installed. Once the command has been executed, the broker should be running on your Pi. To confirm that, just run the following command:
sudo systemctl status mosquitto
This should produce an output on the terminal window similar to what is shown in the following screenshot:
Figure 1.44 – The output of the status command for the broker
The most important thing is that the Active
option should show the active (running)
status, which will verify that our broker is up and running!
Important Note
If the status command shows an output that says that your process is dead and your MQTT broker stopped, restart the MQTT service by typing the following command:
sudo service mosquitto restart
Now, recheck the status, and it should show the status of your MQTT broker as running!
This marks the conclusion of this section. We successfully set up our Raspberry Pi as a local MQTT broker. Additionally, we installed a package that will let us use the Pi as an MQTT client as well. In the next section, we will test our MQTT broker’s functionality through a short demonstration.
Now that the MQTT broker is running on the Raspberry Pi, we will test the connection using a straightforward project. First, open two terminal windows on your Raspberry Pi. Now, we will do the following:
test/message
To do this, you will require the mosquitto-clients
package. Now, type the following command in the terminal:
mosquitto_sub -v -t test/message
This will subscribe to the topic entered after -t
.
Just imagine the two terminals as different MQTT clients where one client is a subscriber to the topics which the other client publishes. To publish a test message of 'Hello World!'
, type the following command:
mosquitto_pub -t test/message -m 'Hello World!'
'Hello World!'
on the terminal along with the topic name, as illustrated in the following screenshots: Figure 1.45 – Raspberry Pi local MQTT test
Congratulations! You have successfully set up your Raspberry Pi. This marks the end of the first chapter of this book. Now, let us summarize what we covered in this chapter.
This chapter introduced us to Raspberry Pi and MQTT. We started with a brief introduction about MQTT.
Next, we covered the Raspberry Pi, the main hardware we will be using throughout this book. We started by covering the hardware specifications and some popular operating systems available for the Pi. Then we set up our Raspberry Pi to work as a local MQTT broker. To do that, we set up the SD card we will be using for our Pi by formatting it and then flashing the latest version of Raspberry Pi onto it. After that, we did the essential steps to set up our Raspberry Pi for the very first time. Next, we installed the packages that let us use our Pi as an MQTT broker as well as an MQTT client. Finally, we tested the functionality of our Pi as a local MQTT broker.
In the next chapter, we will dive deeper into the different clients we can use with our broker. We will set up an MQTT client on our main computer to understand how MQTT actually works.
Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.
If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.
Please Note: Packt eBooks are non-returnable and non-refundable.
Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:
If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:
Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.
You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.
Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.
When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.
For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.