Building the C++ Building Blocks for Low Latency Applications

In the previous chapter, we had a detailed and highly technical discussion of how to approach developing low latency applications in C++. We also investigated the technical details of the C++ programming language as well as the GCC compiler. Now, we will move from a theoretical discussion to building some practical low latency C++ components ourselves.

We will build some relatively general components that can be used in a variety of different low latency applications, such as the ones we discussed in the previous chapters. As we build these basic building blocks in this chapter, we will learn about using C++ effectively to write highly performant C++ code. We will use these components in the rest of the book to demonstrate where these components fit into the electronic trading ecosystem that we will design and build.

In this chapter, we will cover the following topics:

• C++ threading for multi-threaded low latency...

Technical requirements

All the code for this book can be found in the GitHub repository for this book at https://github.com/PacktPublishing/Building-Low-Latency-Applications-with-CPP. The source for this chapter is in the Chapter4 directory in the repository.

We expect you to have at least intermediate C++ programming experience, since we will assume you understand the widely used C++ programming features well. We also assume that you have some experience with network programming in C++, since network programming is a huge topic and cannot be covered in this book. For this book, starting with this chapter, we will use the CMake and Ninja build systems, so we expect you to either understand CMake, g++, Ninja, Make, or some such build system to be able to build the code samples for this book.

The specifications of the environment in which the source code for this book was developed are shown here. We present the details of this environment since all the C++ code presented in this...

C++ threading for multi-threaded low latency applications

The first component we will build is a very small one but still quite fundamental. This section will design and implement a method of creating and running threads of execution. These will be used in many different parts of a full low-latency system, depending on the design of the different sub-components in the system. Depending on the design of the system, different components might work together as a pipeline to facilitate parallel processing. We will use the multi-threading framework in exactly such a way in our electronic trading systems. Another use case is to pass off non-critical tasks such as logging onto disk, computing statistics, and so on to a background thread.

Before we move on to the source code that creates and manipulates threads, let us first quickly define a few useful macros. We will use these functions in many places in the source code that we will be writing in this book, starting with this chapter.

...

Designing C++ memory pools to avoid dynamic memory allocations

We have had several discussions on dynamic memory allocation, the steps the OS needs to perform, and why dynamic memory allocation is slow. Dynamic memory allocation is so slow in fact that low latency applications actively try to avoid it as much as possible on the critical path. We cannot build useful applications without creating and deleting many objects at runtime, and dynamic memory allocation is too slow for low latency applications.

Understanding the definition of a memory pool

First, let us formally define what a memory pool is and why we need one. Many applications (including low latency applications) need to be able to handle many objects and an unknown number of objects. By an unknown number of objects, we mean that the expected count of objects cannot be determined ahead of time, and it cannot be ascertained what the maximum number of objects will be. Obviously, the maximum number of objects possible...

Transferring data using lock-free queues

In the C++ threading for multi-threaded low latency applications section, we hinted that one possible application of having multiple threads is to set up a pipelined system. Here, one component thread performs part of the processing and forwards the results to the next stage of the pipeline for further processing. We will be using such a design in our electronic trading system, but there’ll be more on that later.

Communicating between threads and processes

There are a lot of options when it comes to transferring data between processes and/or threads. Inter-Process Communication (IPC), such as mutexes, semaphores, signals, memory-mapped files, and shared memory, can be used for these purposes. It also gets tricky when there is concurrent access to shared data and the important requirement is to avoid data corruption. Another important requirement is to make sure that the reader and writer have consistent views of the shared data...

Building a low latency logging framework

Now, we will build a low latency logging framework using some of the components we just built in the previous sections. Logging is an important part of any application, whether it is logging general application behavior, warnings, errors, or even performance statistics. However, a lot of important logging output is actually from performance-critical components that are on a critical path.

A naïve logging approach would be to output to the screen, while a slightly better approach would be for logs to be saved to one or more log files. However, here we have a few problems – disk I/O is extremely slow and unpredictable, and string operations and formatting themselves are slow. For these reasons, performing these operations on a performance-critical thread is a terrible idea, so we will build a solution in this section to alleviate the downsides while preserving the ability to output logs as needed.

Before we jump into the logger...

C++ network programming using sockets

In this final section, we will build the last of our basic building blocks – a framework to handle network programming using Unix sockets. We will use this framework to build a server that listens for incoming TCP connections and a client that is capable of establishing a TCP connection to such a server. We will also use this framework to publish UDP traffic and consume from a stream of multicast traffic. Note that to limit the scope of this discussion, we will only discuss Unix sockets without any kernel bypass capabilities. Using kernel bypass and leveraging the kernel bypass API provided by the Network Interface Cards (NICs) that support it is outside the scope of this book. Note also that we expect you to have some basic knowledge or experience with network sockets and, ideally, programming network sockets in C++.

Building a basic socket API

Our goal here is to create a mechanism to create a network socket and initialize it with...

Summary

In this chapter, we jumped into the world of low latency application C++ development. We built some relatively fundamental but extremely useful building blocks that can be used for a variety of low latency application purposes. We put into practice a lot of the theoretical discussions related to using C++ and computer architecture features effectively to build low latency and highly performant applications.

The first component was used to create new threads of execution and run the functions that different components might require. One important functionality here is being able to control the CPU core that the newly created thread gets pinned to by setting the thread affinity.

The second component we built was meant to avoid dynamic memory allocation on the critical code path. We reiterated the inefficiencies associated with dynamic memory allocation and designed a memory pool to be used to pre-allocate memory from the heap when constructed. Then, we added utility to...