Need IPC? Get Your Minions to Work

In the previous chapter, you learned how to send information across threads of the same process. In this chapter, you will look at how to share data between threads of different processes. We will even share information between applications running on different physical computers. We will enhance the Mandelbrot generator application from Chapter 9, Keeping Your Sanity with Multithreading. The Mandelbrot application will now only display results processed by the worker programs. These minions have only one mission: compute the tasks as fast as possible and return a result to your main application.

The following topics will be covered in this chapter:

• Inter-process communication techniques
• Architecturing an IPC project
• Laying down the foundations with an SDK
• Working with QDataStream and QTcpSocket
• Building your own QTcpServer

...

Inter-process communication techniques

An Inter-Process Communication (IPC) is a communication between two or more processes. They can be instances of the same application or different applications. The Qt framework provides multiple modules to help you implement a communication between your applications. Most of these modules are cross-platform. Let's talk about the IPC tools that can be used in a Qt application.

The first tools are the TCP/IP sockets. They provide a bidirectional data exchange over a network. Therefore, you can use them to talk with processes on different computers. Moreover, the loopback interface allows you to communicate with processes running on the same computer. All the required classes are inside the QtNetwork module. This technique relies on a client-server architecture. Here is an example of the server part:

QTcpServer* tcpServer = new QTcpServer...

Architecturing an IPC project

The Mandelbrot picture generator from Chapter 9, Keeping Your Sanity with Multithreading, uses all the cores of your computer to speed up the computing. This time, we want to use all the cores of all of your computers! The first thing is to choose an appropriated IPC technique. For this project example, we want to establish communication between several clients acting as workers to a server running the main application. The TCP/IP sockets allow a one-to-many communication. Moreover, this IPC method is not bound to a single computer and can operate through a network on multiple computers. This project example uses sockets by implementing a multithreaded TCP server.

The following diagram describes the architecture:

Let's talk about the global role of each actor:

• mandelbrot-app: This is the main application displaying the Mandelbrot picture and...

Laying down the foundations with an SDK

The first step is to implement the classes that will be shared between our application and the workers. To do so, we are going to rely on a custom SDK. If you need to refresh your memory about this technique, take a look at Chapter 8, Animations – It's Alive, Alive!.

As a reminder, here is the diagram that describes the SDK:

Let's look at the job of each of these components:

• The Message component encapsulates a piece of information that is exchanged between the application and the worker.
• The JobRequest component contains the necessary information to dispatch a proper job to a worker.
• The JobResult component contains the result of the Mandelbrot set calculation for a given line.
• The MessageUtils component contains helper functions to serialize/deserialize data across the TCP socket.

All these files have to be accessible...

Working with QDataStream and QTcpSocket

The missing piece of the SDK is MesssageUtils. It deserves a dedicated section because it covers two major topics: serialization and QDataStream transactions.

We will start with the serialization. We already know that Message stores only an opaque QByteArray data member. As a consequence, the desired data has to be serialized as QByteArray before being passed to Message.

If we take the example of a JobRequest object, it is not directly sent. We first put it in a generic Message object with the appropriate Message type. The following diagram summarizes the sequence of actions to be done:

The JobRequest object is first serialized to a QByteArray class. It is then passed to a Message instance, which is in turn serialized to a final QByteArray. The deserialization process is the exact opposite of this sequence (from right to left).

Serializing...

Interacting with sockets in the worker

Now that the SDK is completed, we can turn to the worker. The project is complex enough; we can refresh our memory with the mandelbrot-worker architecture:

We will start by implementing the Job class. Inside the mandelbrot-worker project, create a new C++ class named Job. Here is the Job.h content:

#include <QObject> 
#include <QRunnable> 
#include <QAtomicInteger> 
 
#include "JobRequest.h" 
#include "JobResult.h" 
 
class Job : public QObject, public QRunnable 
{ 
    Q_OBJECT 
public: 
    explicit Job(const JobRequest& jobRequest,  
                 QObject *parent = 0); 
    void run() override; 
 
signals: 
    void jobCompleted(JobResult jobResult); 
 
public slots: 
    void abort(); 
 
private: 
    QAtomicInteger<bool> mAbort; 
    JobRequest mJobRequest; 
}; 

If you remember the...

Interacting with sockets from the application

The next project to complete is mandelbrot-app. It will contain QTcpServer, which will interact with the workers and the picture drawing of the Mandelbrot set. As a reminder, the diagram of the mandelbrot-app architecture is shown here:

We will build this application from the ground up. Let's start with the class responsible for maintaining the connection with a specific Worker: WorkerClient. This class will live in its specific QThread and will interact with a Worker class using the same QTcpSocket/QDataStream mechanism we covered in the last section.

In mandelbrot-app, create a new C++ class named WorkerClient and update WorkerClient.h like so:

#include <QTcpSocket> 
#include <QList> 
#include <QDataStream> 
 
#include "JobRequest.h" 
#include "JobResult.h" 
#include "Message.h...

Building your own QTcpServer

Everything is ready to read and write in our sockets. We still need a server to orchestrate all these instances. To do so, we will develop a modified version of the MandelbrotCalculator class, which was covered in Chapter 9, Keeping Your Sanity with Multithreading.

The idea is to respect the same interface, in order to keep MandelbrotWidget oblivious to the fact that the Mandelbrot picture-generation is deported on different processes/machines.

The main difference between the old MandelbrotCalculator and the new one is that we replaced the QThreadPool class with QTcpServer. The MandelbrotCalculator class now only has the responsibility to dispatch JobRequests to Workers and aggregate the results. It will no longer interact with a QThreadPool class.

Create a new C++ class named MandelbrotCalculator.cpp and update MandelbrotCalculator.h to match this...

Summary

IPC is a fundamental mechanism in computer science. In this chapter, you learned the various techniques offered by Qt to do IPC and how to create an application that uses sockets to interact, send, and receive commands. You took the original mandelbrot-threadpool application to the next level by enabling it to generate pictures on a cluster of machines.

Adding IPC on top of a multithreaded application brings some issues. You have many more possible bottlenecks, chances of leaking memory, and having an inefficient calculation. Qt provides multiple mechanisms to do IPC. Since Qt 5.7, the addition of transactions that makes the serialization/deserialization part much easier.

In the next chapter, you will discover the Qt Multimedia framework and how to save and load a C++ object from a file. The project example will be a virtual drum machine. You will be able to save and load...