Leveraging Threading and Concurrency

Most computers contain multiple processors or at least multiple cores, and leveraging this computational power is key to many categories of applications. Unfortunately, many developers still have a mindset of sequential code execution, even though operations that do not depend on each other could be executed concurrently. This chapter presents standard library support for threads, asynchronous tasks, and related components, as well as some practical examples at the end.

Most modern processors (except those dedicated to types of applications that do not require great computing power, such as Internet of Things applications) have two, four, or more cores that enable you to concurrently execute multiple threads of execution. Applications must be explicitly written to leverage the multiple processing units that exist; you can write such applications by executing functions on multiple threads at the same time. Since C++11, the standard library provides...

Working with threads

A thread is a sequence of instructions that can be managed independently by a scheduler, such as the operating system. Threads could be software or hardware. Software threads are threads of execution that are managed by the operating system. They can run on single processing units, usually by time slicing. This is a mechanism where each thread gets a time slot of execution (in the range of milliseconds) on the processing unit before the operating system schedules another software thread to run on the same processing unit. Hardware threads are threads of execution at the physical level. They are, basically, a CPU or a CPU core. They can run simultaneously, that is, in parallel, on systems with multiprocessors or multicores. Many software threads can run concurrently on a hardware thread, usually by using time slicing. The C++ library provides support for working with software threads. In this recipe, you will learn how to create and perform operations with threads...

Synchronizing access to shared data with mutexes and locks

Threads allow you to execute multiple functions at the same time, but it is often necessary that these functions access shared resources. Access to shared resources must be synchronized so that only one thread can read or write from or to the shared resource at a time. In this recipe, we will see what mechanisms the C++ standard defines for synchronizing thread access to shared data and how they work.

Getting ready

The mutex and lock classes discussed in this recipe are available in the std namespace in the <mutex> header, and, respectively, <shared_mutex> for C++14 shared mutexes and locks.

How to do it...

Use the following pattern for synchronizing access with a single shared resource:

1. Define a mutex in the appropriate context (class or global scope):

   std::mutex g_mutex;
   

2. Acquire a lock on this mutex before accessing the shared resource in each thread:

   void...

Finding alternatives for recursive mutexes

The standard library provides several mutex types for protecting access to shared resources. std::recursive_mutex and std::recursive_timed_mutex are two implementations that allow you to use multiple locking in the same thread. A typical use for a recursive mutex is to protect access to a shared resource from a recursive function. A std::recursive_mutex class may be locked multiple times from a thread, either with a call to lock() or try_lock(). When a thread locks an available recursive mutex, it acquires its ownership; as a result of this, consecutive attempts to lock the mutex from the same thread do not block the execution of the thread, creating a deadlock. The recursive mutex is, however, released only when an equal number of calls to unlock() are made. Recursive mutexes may also have a greater overhead than non-recursive mutexes. For these reasons, when possible, they should be avoided. This recipe presents a use case for transforming...

Handling exceptions from thread functions

In the first recipe, we introduced the thread support library and saw how to do some basic operations with threads. In that recipe, we briefly discussed exception handling in thread functions and mentioned that exceptions cannot be caught with a try…catch statement in the context where the thread was started. On the other hand, exceptions can be transported between threads within a std::exception_ptr wrapper. In this recipe, we will see how to handle exceptions from thread functions.

Getting ready

You are now familiar with the thread operations we discussed in the previous recipe, Working with threads. The exception_ptr class is available in the std namespace, which is in the <exception> header; mutex (which we discussed in more detail previously) is also available in the same namespace but in the <mutex> header.

How to do it...

To properly handle exceptions thrown in a worker thread from the main thread...

Sending notifications between threads

Mutexes are synchronization primitives that can be used to protect access to shared data. However, the standard library provides a synchronization primitive, called a condition variable, that enables a thread to signal to others that a certain condition has occurred. The thread or threads that are waiting on the condition variable are blocked until the condition variable is signaled or until a timeout or a spurious wakeup occurs. In this recipe, we will see how to use condition variables to send notifications between thread-producing data and thread-consuming data.

Getting ready

For this recipe, you need to be familiar with threads, mutexes, and locks. Condition variables are available in the std namespace in the <condition_variable> header.

How to do it...

Use the following pattern for synchronizing threads with notifications on condition variables:

1. Define a condition variable (in the appropriate context): ...

Using promises and futures to return values from threads

In the first recipe of this chapter, we discussed how to work with threads. You also learned that thread functions cannot return values and that threads should use other means, such as shared data, to do so; however, for this, synchronization is required. An alternative to communicating a return value or an exception with either the main or another thread is using std::promise. This recipe will explain how this mechanism works.

Getting ready

The promise and future classes used in this recipe are available in the std namespace in the <future> header.

How to do it...

To communicate a value from one thread to another through promises and futures, do this:

1. Make a promise available to the thread function through a parameter; for example:

   void produce_value(std::promise<int>& p)
   {
     // simulate long running operation
     {
       using namespace std::chrono_literals;
       std::this_thread...

Executing functions asynchronously

Threads enable us to run multiple functions at the same time; this helps us take advantage of the hardware facilities in multiprocessor or multicore systems. However, threads require explicit, lower-level operations. An alternative to threads is tasks, which are units of work that run in a particular thread. The C++ standard does not provide a complete task library, but it enables developers to execute functions asynchronously on different threads and communicate results back through a promise-future channel, as seen in the previous recipe. In this recipe, we will see how to do this using std::async() and std::future.

Getting ready

For the examples in this recipe, we will use the following functions:

void do_something()
{
  // simulate long running operation
  {
    using namespace std::chrono_literals;
    std::this_thread::sleep_for(2s);
  } 
  std::lock_guard<std::mutex> lock(g_mutex);
  std::cout << "operation 1 done...

Using atomic types

The thread support library offers functionalities for managing threads and synchronizing access to shared data with mutexes and locks, and, as of C++20, with latches, barriers, and semaphores. The standard library provides support for the complementary, lower-level atomic operations on data, which are indivisible operations that can be executed concurrently from different threads on shared data, without the risk of producing race conditions and without the use of locks. The support it provides includes atomic types, atomic operations, and memory synchronization ordering. In this recipe, we will see how to use some of these types and functions.

Getting ready

All the atomic types and operations are defined in the std namespace in the <atomic> header.

How to do it...

The following are a series of typical operations that use atomic types:

• Use the std::atomic class template to create atomic objects that support atomic operations, such...

Implementing parallel map and fold with threads

In Chapter 3, Exploring Functions, we discussed two higher-order functions: map, which applies a function to the elements of a range by either transforming the range or producing a new range, and fold (also referred to as reduce), which combines the elements of a range into a single value. The various implementations we did were sequential. However, in the context of concurrency, threads, and asynchronous tasks, we can leverage the hardware and run parallel versions of these functions to speed up their execution for large ranges, or when the transformation and aggregation are time-consuming. In this recipe, we will see a possible solution for implementing map and fold using threads.

Getting ready

You need to be familiar with the concepts of the map and fold functions. It is recommended that you read the Implementing higher-order functions map and fold recipe from Chapter 3, Exploring Functions. In this recipe, we will use the...

Implementing parallel map and fold with tasks

Tasks are a higher-level alternative to threads for performing concurrent computations. std::async() enables us to execute functions asynchronously, without the need to handle lower-level threading details. In this recipe, we will take the same task of implementing a parallel version of the map and fold functions, as in the previous recipe, but we will use tasks and see how it compares with the thread version.

Getting ready

The solution presented in this recipe is similar in many aspects to the one that uses threads in the previous recipe, Implementing parallel map and fold with threads. Make sure you read that one before continuing with the current recipe.

How to do it...

To implement a parallel version of the map function, do the following:

1. Define a function template that takes a begin and end iterator for a range and a function to apply to all the elements:

   template <typename Iter, typename F>...

Implementing parallel map and fold with standard parallel algorithms

In the previous two recipes, we implemented parallel versions of the map and fold functions (which are called std::transform() and std::accumulate() in the standard library) using threads and tasks. However, these implementations required manual handling of parallelization details, such as splitting data into chunks to be processed in parallel and creating threads or tasks, synchronizing their execution, and merging the results.

In C++17, many of the standard generic algorithms have been parallelized. In fact, the same algorithm can execute sequentially or in parallel, depending on a provided execution policy. In this recipe, we will learn how to implement map and fold in parallel with standard algorithms.

Getting ready

Before you continue with this recipe, it is recommended that you read the previous two to make sure you understand the differences between various parallel implementations.

How to...

Using joinable threads and cancellation mechanisms

The C++11 class std::thread represents a single thread of execution and allows multiple functions to execute concurrently. However, it has a major inconvenience: you must explicitly invoke the join() method to wait for the thread to finish execution. This can lead to problems because if a std::thread object is destroyed while it is still joinable, then std::terminate() is called. C++20 provides an improved thread class called std::jthread (from joinable thread) that automatically calls join() if the thread is still joinable when the object is destroyed. Moreover, this type supports cancellation through std::stop_source/std::stop_token and its destructor also requests the thread to stop before joining. In this recipe, you will learn how to use these new C++20 types.

Getting ready

Before you continue with this, you should read the first recipe of this chapter, Working with threads, to make sure you are familiar with std::thread...

Synchronizing threads with latches, barriers, and semaphores

The thread support library from C++11 includes mutexes and condition variables that enable thread-synchronization to shared resources. A mutex allows only one thread of multiple processes to execute, while other threads that want to access a shared resource are put to sleep. Mutexes can be expensive to use in some scenarios. For this reason, the C++20 standard features several new, simpler synchronization mechanisms: latches, barriers, and semaphores. Although these do not provide new use cases, they are simpler to use and can be more performant because they may internally rely on lock-free mechanisms.

Getting ready

The new C++20 synchronization mechanisms are defined in new headers. You have to include <latch> for std::latch, <barrier>, or std::barrier, and <semaphore> for std::counting_semaphore and std::binary_semaphore.

The code snippets in this recipe will use the following two functions...

Synchronizing writing to output streams from multiple threads

std::cout is a global object of the std::ostream type. It is used to write text to the standard output console. Although writing to it is guaranteed to be thread-safe, this applies to just one invocation of the operator<<. Multiple such sequenced calls to operator<< can be interrupted and resumed later, making it necessary to employ synchronization mechanisms to avoid corrupted results. This applies to all scenarios where multiple threads operate on the same output stream. To simplify this scenario, C++20 introduced std::basic_osyncstream to provide a mechanism to synchronize threads writing to the same output stream. In this recipe, you will learn how to use this new utility.

How to do it…

To synchronize access to an output stream for writing from multiple threads, do the following:

• Include the <syncstream> header.
• Define a variable of the std::osyncstream type to wrap...