In this chapter, we will start our journey toward using the preferred IoT publish-subscribe lightweight messaging protocol in diverse IoT solutions, combined with mobile apps and web applications. We will learn how MQTT and its lightweight messaging system work.

We will understand the MQTT puzzle: clients, servers (formerly known as brokers), and connections. We will learn the procedures to install an MQTT 3.1.1 Mosquitto server in Linux, macOS, and Windows. We will learn special considerations for running a Mosquitto server in the cloud (Azure, AWS, and other cloud providers). We will gain an understanding of the following:

• Understanding convenient scenarios for the MQTT protocol
• Working with the publish-subscribe pattern
• Working with message filtering
• Understanding the MQTT puzzle: clients, servers, and connections
• Installing a Mosquitto server on Linux
• Installing a Mosquitto server on macOS
• Installing a Mosquitto server on Windows
• Considerations for running a Mosquitto server in the cloud

Imagine that we have dozens of different devices that must exchange data between themselves. These devices have to request data from other devices, and the devices that receive the requests must respond with that data. The devices that request the data must process the data received from the devices that responded with the required data.

The devices are Internet of Things (IoT) boards that have dozens of sensors wired to them. We have the following IoT boards with different processing powers:

• Raspberry Pi 3 Model B+
• Qualcomm DragonBoard 410c
• Udoo Neo
• BeagleBone Black
• Phytec phyBoard-i.MX7-Zeta
• e-con Systems eSOMiMX6-micro
• MinnowBoard Turbot Quad-Core

Each of these boards has to be able to send and receive data. In addition, we want a web application to be able to send and receive data. We want to send and receive data in near-real time over the internet, and we might face some network problems: our wireless networks are somewhat unreliable and we have some high-latency environments. Some devices have low power, many of them are powered by batteries, and their resources are scarce. In addition, we must be careful with the network bandwidth usage because some of the devices use metered connections.

We can use HTTP requests and build a publish-subscribe model to exchange data between different devices. However, there is a protocol that has been specifically designed to be lighter than the HTTP 1.1 and HTTP/2 protocols. The MQ Telemetry Transport (MQTT) is better suited for a scenario in which many devices have to exchange data between themselves in near-real time over the internet and we need to consume the least network bandwidth possible. This protocol works better than HTTP 1.1 and HTTP/2 when unreliable networks are involved and connectivity is intermittent.

The MQTT protocol is a machine-to-machine (M2M) and IoT connectivity protocol. MQTT is a lightweight messaging protocol that works with a server-based publish-subscribe mechanism and runs on top of TCP/IP (Transmission Control Protocol/Internet Protocol). The following diagram shows the MQTT protocol on top of the TCP/IP stack:

MQTT is lighter than the HTTP 1.1 and HTTP/2 protocols, and therefore it is a very interesting option whenever we have to send and receive data in near-real time with a publish-subscribe model, while requiring the lowest possible footprint. MQTT is very popular in IoT, M2M, and embedded projects, but it is also gaining presence in web applications and mobile apps that require assured messaging and an efficient message distribution. As a summary, MQTT is suitable for the following application domains in which data exchange is required:

• Asset tracking and management
• Automotive telematics
• Chemical detection
• Environment and traffic monitoring
• Field force automation
• Fire and gas testing
• Home automation
• In-Vehicle Infotainment (IVI)
• Medical
• Messaging
• Point of Sale (POS) kiosks
• Railway
• Radio-Frequency Identification (RFID)
• Supervisory Control and Data Acquisition (SCADA)
• Slot machines

As a summary, MQTT was designed to be suitable to support the following typical challenges in IoT, M2M, embedded, and mobile applications:

• Be lightweight to make it possible to transmit high volumes of data without huge overheads
• Distribute minimal packets of data in huge volumes
• Support an event-oriented paradigm with the asynchronous, bidirectional, low-latency push delivery of messages
• Easily emit data from one client to many clients
• Make it possible to listen for events whenever they happen (event-oriented architecture)
• Support always-connected and sometimes-connected models
• Publish information over unreliable networks and provide reliable deliveries over fragile connections
• Work very well with battery-powered devices or require low power consumption
• Provide responsiveness to make it possible to achieve near-real-time delivery of information
• Offer security and privacy for all the data
• Be able to provide the necessary scalability to distribute data to hundreds of thousands of clients

Before we dive deep into MQTT, we must understand the publish-subscribe pattern, also known as the pub-sub pattern. In the publish-subscribe pattern, a client that publishes a message is decoupled from the other client or clients that receive the message. The clients don't know about the existence of the other clients. A client can publish messages of a specific type and only the clients that are interested in that specific type of message will receive the published messages.

The publish-subscribe pattern requires a server, also known as a broker. All the clients establish a connection with the server. A client that sends a message through the server is known as the publisher. The server filters the incoming messages and distributes them to the clients that are interested in that type of received messages. Clients that register to the server as interested in specific types of messages are known as subscribers. Hence, both publishers and subscribers establish a connection with the server.

It is easy to understand how things work with a simple diagram. The following diagram shows one publisher and two subscribers connected to a server:

A Raspberry Pi 3 Model B+ board with an altitude sensor wired to it is a publisher that establishes a connection with the server. A BeagleBone Black board and a Udoo Neo board are two subscribers that establish a connection with the server.

The BeagleBone Black board indicates to the server that it wants to subscribe to all messages that belong to the sensors/drone01/altitude topic. The Udoo Neo board indicates the same to the server. Hence, both boards are subscribed to the sensors/drone01/altitude topic.

The Raspberry Pi 3 Model B+ board publishes a message with 100 feet as the payload and sensors/drone01/altitude as the topic. This board, that is, the publisher, sends the publish request to the server.

The server distributes the message to the two clients that are subscribed to the sensors/drone01/altitude topic: the BeagleBone Black and the Udoo Neo boards.

Publishers and subscribers are decoupled in space because they don't know each other. Publishers and subscribers don't have to run at the same time. The publisher can publish a message and the subscriber can receive it later. In addition, the publish operation isn't synchronized with the receive operation.

A publisher requests the server to publish a message, and the different clients that have subscribed to the appropriate topic can receive the message at different times. The publisher can send a message as an asynchronous operation to avoid being blocked until the server receives the message. However, it is also possible to send a message to the server as a synchronous operation with the server and to continue the execution only after the operation was successful. In most cases, we will want to take advantage of asynchronous operations.

A publisher that requires sending a message to hundreds of clients can do it with a single publish operation to a server. The server is responsible for sending the published message to all the clients that have subscribed to the appropriate topic. Because publishers and subscribers are decoupled, the publisher doesn't know whether any subscriber is going to listen to the messages it is going to send. Hence, sometimes it is necessary to make the subscriber become a publisher too and to publish a message indicating that it has received and processed a message. The specific requirements depend on the kind of solution we are building. MQTT offers many features that make our lives easier in many of the scenarios we have been analyzing. We will use these different features throughout the book.

The server has to make sure that subscribers only receive the messages they are interested in. It is possible to filter messages based on different criteria in a publish-subscribe pattern. We will focus on analyzing topic-based filtering, also known as subject-based filtering.

Consider that each message belongs to a topic. When a publisher requests the server to publish a message, it must specify both the topic and the message. The server receives the message and delivers it to all the subscribers that have subscribed to the topic to which the message belongs.

The server doesn't need to check the payload for the message to deliver it to the corresponding subscribers; it just needs to check the topic for each message that has arrived and needs to be filtered before publishing it to the corresponding subscribers.

A subscriber can subscribe to more than one topic. In this case, the server has to make sure that the subscriber receives messages that belong to all the topics to which it has subscribed. It is easy to understand how things work with another simple diagram.

The following diagram shows two future publishers that haven't published any messages yet, a server, and two subscribers connected to the server:

A Raspberry Pi 3 Model B+ board with an altitude sensor wired to it and a Raspberry Pi 3 board with a temperature sensor wired to it will be the two publishers. A BeagleBone Black board and a Udoo Neo board are the two subscribers that establish a connection to the server.

The BeagleBone Black board indicates to the server that it wants to subscribe to all the messages that belong to the sensors/drone01/altitude topic. The Udoo Neo board indicates to the server that it wants to subscribe to all the messages that belong to either of the following two topics: sensors/drone01/altitude and sensors/drone40/temperature. Hence, the Udoo Neo board is subscribed to two topics while the BeagleBone Black board is subscribed to just one topic.

The following diagram shows what happens after the two publishers connect and publish messages to different topics through the server:

The Raspberry Pi 3 Model B+ board publishes a message with 120 feet as the payload and sensors/drone01/altitude as the topic. The board, that is, the publisher, sends the publish request to the server. The server distributes the message to the two clients that are subscribed to the sensors/drone01/altitude topic: the BeagleBone Black and the Udoo Neo boards.

The Raspberry Pi 3 board publishes a message with 75 F as the payload and sensors/drone40/temperature as the topic. The board, that is, the publisher, sends the publish request to the server. The server distributes the message to the only client that is subscribed to the sensors/drone40/temperature topic: the Udoo Neo board. Thus, the Udoo Neo board receives two messages from the server, one that belongs to the sensors/drone01/altitude topic and one that belongs to the sensors/drone40/temperature topic.

The following diagram shows what happens after one publisher publishes a message to a topic through the server, and this topic has only one subscriber:

The Raspberry Pi 3 board publishes a message with 76 F as the payload and sensors/drone40/temperature as the topic. The board, that is, the publisher, sends the publish request to the server. The server distributes the message to the only client that is subscribed to the sensors/drone40/temperature topic: the Udoo Neo board.

In versions of the MQTT protocol lower than 3.1.1, the MQTT server was known as an MQTT broker. Starting in MQTT 3.1.1, the MQTT broker was renamed the MQTT server, and therefore we will refer to it as the server. However, we must take into account that the documentation for MQTT servers, tools, and client libraries can use the old MQTT broker name to refer to the server. MQTT servers are also known as message brokers.

The MQTT server uses the previously explained topic-based filtering to filter and distribute messages to the appropriate subscribers. There are many MQTT server implementations that provide additional message filtering features by providing custom plugins. However, we will focus on the features that are part of the MQTT protocol requirements.

As previously explained, in MQTT publishers and subscribers are completely decoupled. Publishers and subscribers are MQTT clients that only establish a connection with the MQTT server. An MQTT client can be both a publisher and a subscriber at the same time, that is, the client can publish messages to specific topics and also receive messages that belong to topics to which the client has subscribed.

There are MQTT client libraries available for the most popular programming languages and platforms. One of the most important things that we must consider when we select the MQTT client library is the list of MQTT features that they support and the ones that we need for our solution. Sometimes, we can choose between many libraries for a specific programming language and platform, and some of them might not implement all the features. We will use the most complete libraries for modern Python versions that support a wide range of platforms throughout the book.

Any device that has a TCP/IP stack and is capable of using an MQTT library can become an MQTT client, that is, a publisher, a subscriber, or both a publisher and a subscriber. The MQTT library makes it possible for the device to talk to MQTT on top of TCP/IP and to interact with specific types of MQTT server. For example, any of the following can become an MQTT client, among other devices:

• An Arduino board
• A Raspberry Pi 3 Model B+ board
• A BeagleBone Black board
• A Udoo Neo board
• An iPhone
• An iPad
• An Android tablet
• An Android smartphone
• A laptop running Windows
• A server running Linux
• A MacBook running macOS

There are many MQTT servers available for the most popular platforms, including Linux, Windows, and macOS. Many of them are servers that can work as MQTT servers and they also provide additional futures. An MQTT server might implement only a subset of the MQTT capabilities and might have specific limitations. Hence, it is very important to check all the capabilities we will require in our solution before selecting an MQTT server. As happens with other middleware, we have open source versions, free versions, and paid versions. Thus, we also have to make sure we select the appropriate MQTT server based on our budget and our specific needs.

Throughout this book, we will work with the Eclipse Mosquitto MQTT server (http://www.mosquitto.org). Mosquitto is an open source MQTT server with an EPL/EDL license that is compatible with MQTT versions 3.1.1 and 3.1. We can take advantage of everything we learn with any other MQTT server, such as Erlang MQTT Broker (EMQ), also known as Emqttd (http://www.emqtt.io), and HiveMQ (http://hivemq.com), among others. In addition, we might use our knowledge to work with a cloud-based MQTT server such as CloudMQTT (http://www.cloudmqtt.com) or the PubNub MQTT bridge (http://pubnub.com). We will specifically work with cloud-based MQTT providers too.

The MQTT server is the central hub of the publish-subscribe model we previously analyzed. The MQTT server is responsible for the authentication and authorization of the MQTT clients that will be able to become publishers and/or subscribers after they are authenticated and authorized. So, the first thing that an MQTT client must do is to establish a connection with the MQTT server.

In order to establish a connection, the MQTT client must send a CONNECT control packet to the MQTT server with a payload that must include all the necessary information to initiate the connection and proceed with the authentication and authorization. The MQTT server will check the CONNECT packet, perform authentication and authorization, and send a response to the client with a CONNACK control packet that we will analyze in detail after understanding the CONNECT control packet. If the MQTT client sends an invalid CONNECT control packet, the server will automatically close the connection.

The following diagram shows the interaction between an MQTT client and an MQTT server to establish a connection:

The CONNECT control packet must include values for the following fields in the payload, and bits for a special flags byte that is included in the control packet. We want to understand the meaning of these fields and flags because we will be able to specify their values when we work with MQTT tools and MQTT client libraries in Python:

• ClientId: The client identifier, also known as client ID, is a string that identifies each MQTT client that connects to an MQTT server. Each client that connects to the MQTT server must have a unique ClientId, and the server uses it to identify the state that it holds related to the MQTT session between the client and the server. If a client specifies an empty value as the ClientId, the MQTT server must generate a unique ClientId to identify the client. However, this behavior depends on the value specified for the CleanSession field.

• CleanSession: The clean session flag is a Boolean value that specifies what happens after an MQTT client disconnects from the MQTT server and then reconnects. If CleanSession is set to 1 or True, the client indicates to the MQTT server that the session will only last as long as the network connection is alive. After the MQTT client disconnects from the MQTT server, any information related to the session is discarded. A new connection from the same MQTT client to the MQTT server will not use the data from the previous session and will be a new clean session. If CleanSession is set to 0 or False, we will work with a persistent session. In this case, the MQTT server stores all the subscriptions for the MQTT client and when the MQTT client disconnects, the MQTT server stores all the messages that arrive with specific quality of service levels that match the subscriptions that the MQTT client had at the time the disconnection occurred. This way, when the same MQTT client establishes a new connection with the MQTT server, the MQTT client will have the same subscriptions and will receive all the messages that it couldn't receive when it lost the connection. We will dive deep into quality of service levels for messages and their relationship with the clean session flag or the persistent session option later in Chapter 2, Using Command-Line and GUI Tools to Learn How MQTT Works.

• UserName: If the client wants to specify a username to request authentication and authorization from the MQTT server, it must set the UserName flag to 1 or True and specify a value for the UserName field
• Password: If the client wants to specify a password to request authentication and authorization from the MQTT server, it must set the Password flag to 1 or True and specify a value for the Password field

• ProtocolLevel: The protocol level value indicates the MQTT protocol version that the MQTT client requests the MQTT server to use. Remember, we will always work with MQTT version 3.1.1.
• KeepAlive: KeepAlive is a time interval expressed in seconds. If the value for KeepAlive is not equal to 0, the MQTT client commits to send control packets to the server within the time specified for KeepAlive. If the MQTT client doesn’t have to send any control packets, it must send a PINGREQ control packet to the MQTT server, a ping request to tell the MQTT server that the client connection is alive. The MQTT server responds to the PINGREQ control packet with a PINGRESP response to the MQTT client, a ping response to tell the MQTT client that the connection with the MQTT server is alive. The connection is closed when there is an absence of any of these control packets. If the value for KeepAlive is 0, the keep-alive mechanism is turned off.
• Will, WillQoS, WillRetain, WillTopic, and WillMessage: These flags and fields allow the MQTT client to take advantage of the last will and testament feature of MQTT. If the MQTT client sets the Will flag to 1 or True, it specifies that it wants the MQTT server to store a last will message associated with the session. The WillQoS flag specifies the desired quality of service for the last will message, and the WillRetain flag indicates whether this message must be retained when it is published. If the MQTT client sets the Will flag to 1 or True, it must specify the topic for the Will message and the message in the WillTopic and WillMessage fields. If the MQTT client is disconnected or loses the connection with the MQTT server, the MQTT server will publish the message specified in the WillMessage field to the topic indicated in the WillTopic field with the chosen quality of service. We will analyze this feature in detail later.

The MQTT server will process a valid CONNECT control packet and it will respond with a CONNACK control packet. This control packet will include values for the following flags included in the header. We want to understand the meaning of these flags because we will be able to retrieve their values when we work with MQTT tools and MQTT client libraries:

• SessionPresent: If the MQTT server received a connection request with the CleanSession flag set to 1 or True, the value for the SessionPresent flag will be 0 or False because no stored session will be reused. If the CleanSession flag was set to 0 or False in the connection request, the MQTT server will work with a persistent session, and the value for the SessionPresent flag will be 1 or True if the server had a persistent session for the client from a previous connection and retrieves it. Otherwise, SessionPresent will be 0 or False. The MQTT client that wanted to work with a persistent session can use the value of this flag to determine whether it has to request subscription to the desired topics, or whether the subscriptions have been restored from the persisted session.
• ReturnCode: If the authorization and authentication passed and the connection was successfully established, the value for ReturnCode will be 0. Otherwise, the return code will be different than 0 and the network connection between the client and the server will be closed. The following table shows the possible values for ReturnCode with their meanings:

Now, we will learn the necessary steps to install a Mosquitto server on the most popular operating systems: Linux, macOS, and Windows.

First, we will start with Linux; specifically, we will work with Ubuntu Linux. If you want to work with a different Linux distribution, you can find details about the installation procedure in the Mosquitto downloads section: http://mosquitto.org/download.

Follow these steps to install a Mosquitto server on Ubuntu Linux; take into account that you will require root privileges:

1. Open a Terminal window or use a secure shell to access Ubuntu and run the following command to add the Mosquitto repository:

      sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa

You will see an output similar to the next lines (the temporary filenames will be different):

      gpg: keyring `/tmp/tmpi5yrsz7i/secring.gpg' created
      gpg: keyring `/tmp/tmpi5yrsz7i/pubring.gpg' created
      gpg: requesting key 262C4500 from hkp server keyserver.ubuntu.com
      gpg: /tmp/tmpi5yrsz7i/trustdb.gpg: trustdb created
      gpg: key 262C4500: public key "Launchpad mosquitto" imported
      gpg: Total number processed: 1
      gpg: imported: 1 (RSA: 1)
      OK

2. Run the following command to update the packages with the recently added Mosquitto repository:

      sudo apt-get update

You will see an output similar to the next lines. Note that the next lines show the output of an Ubuntu server running as a Windows Azure virtual machine, and therefore the output will be similar:

      Hit:1 http://azure.archive.ubuntu.com/ubuntu xenial InRelease
      Get:2 http://azure.archive.ubuntu.com/ubuntu xenial-updates       
      InRelease [102 kB]
      Get:3 http://azure.archive.ubuntu.com/ubuntu xenial-backports 
      InRelease [102 kB]

      ...

      Get:32 http://security.ubuntu.com/ubuntu xenial-security/universe        
      Translation-en [121 kB]
      Get:33 http://security.ubuntu.com/ubuntu xenial-
      security/multiverse amd64 Packages [3,208 B]
      Fetched 12.8 MB in 2s (4,809 kB/s)
      Reading package lists... Done

3. Now, run the following command to install the package for the Mosquitto server:

      sudo apt-get install mosquitto

You will see an output similar to the next lines.

4. Enter Y and press Enter to answer the question and complete the installation process:

      Building dependency tree
      Reading state information... Done
      The following additional packages will be installed:
        libev4 libuv1 libwebsockets7
      The following NEW packages will be installed:
        libev4 libuv1 libwebsockets7 mosquitto
      0 upgraded, 4 newly installed, 0 to remove and 29 not upgraded.
      Need to get 280 kB of archives.
      After this operation, 724 kB of additional disk space will be 
      used.
      Do you want to continue? [Y/n] Y

5. The last lines should include a line that says Setting up mosquitto followed by the version number, as shown in the following lines:

      Setting up libuv1:amd64 (1.8.0-1) ...
      Setting up libev4 (1:4.22-1) ...
      Setting up libwebsockets7:amd64 (1.7.1-1) ...
      Setting up mosquitto (1.4.15-0mosquitto1~xenial1) ...
      Processing triggers for libc-bin (2.23-0ubuntu10) ...
      Processing triggers for systemd (229-4ubuntu21.1) ...
      Processing triggers for ureadahead (0.100.0-19) ...

6. Now, run the following command to install the Mosquitto client packages that will allow us to run commands to publish messages to topics and subscribe to topic filters:

      sudo apt-get install mosquitto-clients

You will see an output similar to the next lines.

7. Enter Y and press Enter to answer the question and complete the installation process:

      Reading package lists... Done
      Building dependency tree
      Reading state information... Done
      The following additional packages will be installed:
        libc-ares2 libmosquitto1
      The following NEW packages will be installed:
        libc-ares2 libmosquitto1 mosquitto-clients
      0 upgraded, 3 newly installed, 0 to remove and 29 not upgraded.
      Need to get 144 kB of archives.
      After this operation, 336 kB of additional disk space will be   
      used.
      Do you want to continue? [Y/n] Y

The last lines should include a line that says Setting up mosquitto-clients followed by the version number, as shown in the following lines:

      Setting up libmosquitto1:amd64 (1.4.15-0mosquitto1~xenial1) ...
      Setting up mosquitto-clients (1.4.15-0mosquitto1~xenial1) ...
      Processing triggers for libc-bin (2.23-0ubuntu10) ...

8. Finally, run the following command to check the status for the mosquitto service that was recently installed:

      sudo service mosquitto status

The first lines of the output should be similar to the following lines with an active (running) status displayed. The details after CGroup indicate the command line that started the service. The -c option followed by /etc/mosquitto/mosquitto.conf specifies that Mosquitto is using this configuration file:

mosquitto.service - LSB: mosquitto MQTT v3.1 message broker
   Loaded: loaded (/etc/init.d/mosquitto; bad; vendor preset: enabled)
   Active: active (running) since Sun 2018-03-18 19:58:15 UTC; 3min 8s ago
     Docs: man:systemd-sysv-generator(8)
   CGroup: /system.slice/mosquitto.service
           └─15126 /usr/sbin/mosquitto -c /etc/mosquitto/mosquitto.conf

You can also run the following command to check whether the Mosquitto MQTT server is listening at the default port, 1883:

netstat -an | grep 1883

The following lines show the results of the previous command that indicate the Mosquitto MQTT server has opened an IPv4 and an IPv6 listen socket on port 1883:

tcp 0 0 0.0.0.0:1883 0.0.0.0:* LISTEN

tcp6 0 0 :::1883 :::* LISTEN

Follow these steps to install a Mosquitto server on macOS, known as OS X prior to macOS Sierra:

1. If you don't have Homebrew installed, open a Terminal window and run the command indicated on the Homebrew homepage, http://brew.sh, to install this popular package manager for macOS. The following command will do the job. However, it is convenient to check the Homebrew homepage and check all the detailed instructions that are always updated with the newest versions of macOS that become available. If you already have Homebrew installed, move to the next step:

      /usr/bin/ruby -e "$(curl -fsSL      
    https://raw.githubusercontent.com/Homebrew/install/master/install)"

2. Open a Terminal window and run the following command to request Homebrew to install Mosquitto:

      brew install mosquitto

Notice that in some cases, Homebrew might require additional software to be installed on your computer before you can install Mosquitto. If it is necessary to install additional software, such as Xcode command-line tools, Homebrew will provide you with the necessary instructions.

3. The following lines show the last messages shown in Terminal that indicate that Homebrew has installed Mosquitto and the instructions to start the MQTT server:

      ==> Installing dependencies for mosquitto: c-ares, openssl,      
      libev, libuv, libevent, libwebsockets
      ==> Installing mosquitto dependency: c-ares
      ==> Caveats
      A CA file has been bootstrapped using certificates from the 
      SystemRoots
      keychain. To add additional certificates (e.g. the certificates  
      added in the System keychain), place .pem files in
      /usr/local/etc/openssl/certs and run
      /usr/local/opt/openssl/bin/c_rehash

      This formula is keg-only, which means it was not symlinked into 
      /usr/local, because Apple has deprecated use of OpenSSL in favor  
      of its own TLS and crypto libraries. If you need to have this           
      software first in your PATH run:
      echo 'export PATH="/usr/local/opt/openssl/bin:$PATH"' >>         
      ~/.bash_profile

      For compilers to find this software you may need to set:
      LDFLAGS: -L/usr/local/opt/openssl/lib
      CPPFLAGS: -I/usr/local/opt/openssl/include

      ==> Installing mosquitto
      ==> Downloading https://homebrew.bintray.com/bottles/mosquitto-        
          1.4.14_2.el_capit
       ##################################################
       #####################100.0%
      ==> Pouring mosquitto-1.4.14_2.el_capitan.bottle.tar.gz
      ==> Caveats
      mosquitto has been installed with a default configuration file.
      You can make changes to the configuration by editing:
          /usr/local/etc/mosquitto/mosquitto.conf

      To have launchd start mosquitto now and restart at login:
        brew services start mosquitto

      Or, if you don't want/need a background service you can just run:
        mosquitto -c /usr/local/etc/mosquitto/mosquitto.conf

4. After the Mosquitto installation has been completed, run the following command in a new Terminal window to launch Mosquitto with the default configuration file. The -c option followed by /usr/local/etc/mosquitto/mosquitto.conf specifies that we want to use this configuration file:

      /usr/local/sbin/mosquitto -c       
     /usr/local/etc/mosquitto/mosquitto.conf

The following is the output after you run the previous command:

     1521488973: mosquitto version 1.4.14 (build date 2017-10-22    
     16:34:20+0100) starting
     1521488973: Config loaded from 
     /usr/local/etc/mosquitto/mosquitto.conf.
     1521488973: Opening ipv4 listen socket on port 1883.
     1521488973: Opening ipv6 listen socket on port 1883.

The last lines indicate the Mosquitto MQTT server has opened an IPv4 and an IPv6 listen socket on the default TCP port, 1883. Leave the Terminal window opened, because we need Mosquitto running on the local computer to work with the next examples.

Follow these steps to install a Mosquitto server on Windows. Take into account that you will require Windows Vista or higher (Windows 7, 8, 8.1, 10, or greater). The instructions also work on Windows Server 2008, 2012, 2016, or greater:

1. Download the executable file listed under Binary Installation and Windows that provides the native build on the Mosquitto downloads web page: http://mosquitto.org/download. For Mosquitto 1.4.15, the filename is mosquitto-1.4.15-install-win32.exe. You must click or tap on the filename and you will be redirected to the Eclipse repository with many mirror options, including a default recommendation, from which you can download the executable file.
2. Run the previously downloaded executable file and the mosquitto Setup Wizard will display its welcome dialog box. Click Next > to continue. The setup wizard will display the dependencies that you must install: OpenSSL and pthreads. The dialog box will display the links that you can use to download and run installers for these two requirements, as shown in the following screenshot:

3. If you don't have Win32 OpenSSL v1.0.2j Light installed on Windows, go to the Win32 OpenSSL web page, http://slproweb.com/products/Win32OpenSSL.html, and download the Win32 OpenSSL v1.1.0g Light file listed in a table under Download Win32 OpenSSL. Do not download the Win64 version, because you will need the Win32 version to make it possible for Mosquitto to have its dependencies. If you already have Win32 OpenSSL v1.1.0g Light installed, advance to step 7. For Win32 OpenSSL v1.1.0g Light, the filename is Win32OpenSSL_Light-1_1_0g.exe. Run the downloaded executable file and OpenSSL Light (32-bit) will display its welcome dialog box. Click Next > to continue.
4. The setup wizard will display the License Agreement. Read it, select I Accept the agreement, and click Next >. Select the folder in which you want to install OpenSSL Light (32-bit) if you don't want to use the default folder. Remember the folder you specify, because you will need to copy a few DLL files from this folder later. The default folder is C:\OpenSSL-Win32.
5. Click Next > to continue, specify a different Start Menu folder if necessary, and click Next >. Select The OpenSSL binaries (/bin) directory as the desired option for Copy OpenSSL DLLs to. This way, the installation will copy the DLLs to a bin subfolder within the previously specified folder, by default C:\OpenSSL-Win32\bin.
6. Click Next > to continue. Review the selected installation options and click Install to complete the installation for OpenSSL Light (32-bit). Finally, consider a donation to the Win32 OpenSSL project and click Finish to exit setup.
7. Go to the following address in your web browser: ftp://sources.redhat.com/pub/pthreads-win32/dll-latest/dll/x86. The browser will display many files for this FTP directory. Right-click on pthreadVC2.dll and save the file in your Downloads folder. You will have to copy this DLL to the Mosquitto installation folder later.
8. Now, go back to the mosquitto Setup window and click Next > to continue. By default, Mosquitto will install the files and the Mosquitto Service. Leave the default components to install those selected and click Next > to continue.
9. Select the folder in which you want to install Mosquitto if you don't want to use the default folder. Remember the folder you specify, because you will need to copy a few DLL files to this folder later. The default folder is C:\Program Files (x86)\mosquitto. Click Install to complete the installation. Notice that the mosquitto Setup Wizard might display errors related to missing DLLs. We will fix this issue in the next steps. Once the installation has completed, click Finish to close the mosquitto Setup Wizard.

10. Open a File Explorer window and go to the bin subfolder within the folder in which you installed OpenSSL Light (32-bit), by default C:\OpenSSL-Win32\bin.
11. Copy the following four DLLs: libcrypto-1_1.dll, libeay32.dll, ssleay32.dll, and libssl-1_1.dll. Now, go to the folder in which you installed Mosquitto and paste these four DLLs. By default, the Mosquitto installation folder is C:\Program Files (x86)\mosquitto. You will need to provide administrator permission to paste the DLLs in the default folder.
12. Open a File Explorer window and go to your Downloads folder. Copy the pthreads DLL you downloaded in one of the previous steps, pthreadVC2.dll. Now, go to the folder in which you installed Mosquitto and paste this DLL. You will need to provide administrator permission to paste the DLL into the default Mosquitto installation folder.
13. Now that all the dependencies are included in the Mosquitto installation folder, it is necessary to run the installation again to make the Mosquitto setup configure the Windows service. Run the previously downloaded Mosquitto installation executable again. For Mosquitto 1.4.15, the filename is mosquito-1.4.15-install-win32.exe. Make sure you specify the same installation folder as the folder in which you copied the DLLs and that the Service component is activated. Click Next > many times and click Install to complete the configuration for the Windows service. Once the installation has completed, click Finish to close the mosquitto Setup Wizard.
14. Open the Services application in Windows and search for the service whose name is Mosquitto Broker. Right-click on the service name and select Start. The Status will change to Running. By default, the service is configured to have its Startup Type set to Automatic. If you don't want to automatically start the Mosquitto Broker service, change the Startup Type to Manual. You will have to repeat the step you just did to manually start the service before working with Mosquitto on a Windows computer. Notice that the description for the service is MQTT v3.1 broker, as shown in the following screenshot. The description is outdated because the service provides an MQTT 3.1.1 server that is also compatible with MQTT 3.1:

Open a Windows PowerShell or a Command Prompt window and run the following command to check whether the Mosquitto MQTT server is listening at the default TCP port, 1883:

     netstat -an | findstr 1883

The following lines show the results of the previous command that indicate the Mosquitto MQTT server has opened an IPv4 and an IPv6 listen socket on port 1883:

    TCP 0.0.0.0:1883 0.0.0.0:0 LISTENING
    TCP [::]:1883 [::]:0 LISTENING

We have worked with the default configuration for a Mosquitto server in Linux, macOS, and Windows. The Mosquitto server will work with TCP port 1883. If you want to interact with the Mosquitto server from a different device or computer, you have to make sure that the firewall that is running on your computer has the appropriate configuration for this port number.

When you run a Mosquitto server on a Linux or Windows virtual machine in the cloud, you also have to make sure that the virtual machine network filter that controls both inbound and outbound traffic has the appropriate configuration to allow inbound and outbound traffic on port 1883. You must authorize inbound and outbound traffic on port 1883.

Let's see whether you can answer the following questions correctly:

1. MQTT runs on top of:
   1. The MQIP protocol
   2. The TCP/IP protocol
   3. The IoT protocol
2. The data for an MQTT message is known as:
   1. Payload
   2. Packet
   3. Upload
3. In MQTT version 3.1.1, the broker is named:
   1. MQTT agent
   2. MQTT client
   3. MQTT server
4. Mosquitto is:
   1. A cloud-based MQTT server that is only available on Windows Azure
   2. A cloud-based MQTT server that is only available on Amazon Web Services
   3. An open source MQTT server compatible with MQTT versions 3.1.1 and 3.1
5. The default TCP port that a Mosquitto server uses is:
   1. 22
   2. 1883
   3. 9000

The rights answers are included in the Appendix, Solutions.

In this chapter, we started our journey toward understanding the MQTT protocol. We understood convenient scenarios for this protocol, the details of the publish-subscribe pattern, and message filtering. We learned basic concepts related to MQTT and understood the different components: clients, servers or brokers, and connections.

We learned to install a Mosquitto server on Linux, macOS, and Windows. We used the default configuration, as it allowed us to learn how everything works under the hood while using Mosquitto. Then, we will secure the server. This way, it will be easier for us to start using the client libraries in Python to publish MQTT messages and subscribe to MQTT topic filters.

Now that we have our environment ready to start working with a still-unsecured Mosquitto server, we will work with command-line and GUI tools to learn how MQTT works in detail. We will learn MQTT basics, the specific vocabulary for MQTT, and its working modes, which are the topics that we are going to discuss in Chapter 2, Using Command-Line and GUI Tools to Learn How MQTT Works.