12. Concurrency

Overview

In this chapter we will explore Clojure's concurrency features. On the JVM, you will learn the basics of programming with multiple processor threads: starting a new thread and using the results. To coordinate your threads, we will use Clojure's innovative reference types. One of these, the atom, can also be used in a JavaScript environment.

By the end of this chapter, you will be able to build simple browser games and manage their state using atoms.

Introduction

Ever since the Clojure language was first introduced, its concurrency model has been one of its major selling points. In programming, the word "concurrency" can apply to a lot of different situations. To start with a simple definition, any time your program or your system has more than one simultaneous flow of operations, you are dealing with concurrency. In multithreaded Java programs, that would mean code running simultaneously in separate processor threads. Each processor thread follows its own internal logic, but to work properly your program needs to coordinate the communication between the different threads. Even though JavaScript runtimes are single-threaded, both the browser and Node.js environments have their own ways of dealing with simultaneous logical flows. While the roots of Clojure's concurrency are definitely in Java, some of the ideas and tools apply equally in ClojureScript.

In this chapter, you will learn the basics of concurrent programming...

Concurrency in General

Modern computers use threads to distribute execution between multiple processor cores. There are many reasons for this, including the physics of microchip design and the need for user environments that remain responsive even when one program is performing an intensive computation in the background. Everyone wants to be able to check their email, listen to music, and run their Clojure REPL at the same time! Inside a program, this kind of multitasking can also represent a significant performance gain. While one thread is waiting for data from the disk drive, another for the network, two other threads can be processing data. When done correctly, this can represent a significant gain in performance and overall efficiency. The operative phrase here, though, is "when done correctly." Concurrency can be tricky.

Most computer code is written in a linear manner: do this, do that, then do this. The source code for a method or a function reads from top...

Futures

With pmap, Clojure takes care of all the thread management for you, which makes things easy. A lot of times, however, you need more control over your threads than what pmap provides. Clojure's futures do just this. They are a mechanism for spawning and waiting for new threads.

Consider a situation where two expensive calculations are needed to perform a third operation, such as adding the two results together. In a single-threaded context, we would just write this:

(+ (expensive-calc-1 5) (expensive-calc-2 19))

Written this way, the call to expensive-calc-1 needs to complete before expensive-calc-2 can start. If the calculations could be run in parallel, we would cut the execution time nearly in half, in the best cases. Running the two threads in parallel creates some new problems, though. We need a way of coordinating the return values, especially since we don't know whether expensive-calc-1 or expensive-calc-2 will complete first. We need a way to wait...

Coordination

Futures work well for cases like the crowdspell example. Work is assigned to a thread; the thread performs its task independently and returns a result to the initial thread. The coordination is in the gathering of the results: evaluation is blocked until all the futures have completed. Thanks to immutability, there is a guarantee that simultaneous threads won't interfere with each other because nothing is shared.

This simple model is effective precisely because it is simple. Sometimes, however, more coordination is necessary, especially when communication between threads is required.

With a future, we fork, perform a computation, and return the data:

Figure 12.6: With a future, coordination occurs when the future is dereferenced

Message sending is one way to communicate among threads. Now, we imagine three threads that send messages to one another:

Figure 12.7: Complex interactions between multiple threads

To...

Summary

Concurrency, by its very nature, is a complex problem. While it's impossible to cover all the techniques you might need, hopefully, this chapter will provide you with the tools to get started. We covered the usage of pmap and future for using multiple threads. We also saw Clojure's reference types: var, atoms, agents, and refs. We used atoms to manage state in a browser-based ClojureScript application.

For each of these topics, there is a lot more that can be said. What you learn further down the road will depend on the kinds of problems you need to solve. Concurrency is one of the areas where the problems will be more diverse than almost any other. Familiarity with Clojure's basic approach to these questions will start you in the right direction when you search for solutions.

In the next chapter, we will take another big step toward real-world Clojure by learning how to interact with databases.