Preface

The content in this book was initially written as a series of shorter books for programmers wanting to learn asynchronous programming from the ground up using Rust. I found the existing material I came upon at the time to be in equal parts frustrating, enlightening, and confusing, so I wanted to do something about that.

Those shorter books became popular, so when I got the chance to write everything a second time, improve the parts that I was happy with, and completely rewrite everything else and put it in a single, coherent book, I just had to do it. The result is right in front of you.

People start programming for a variety of different reasons. Scientists start programming to model problems and perform calculations. Business experts create programs that solve specific problems that help their businesses. Some people start programming as a hobby or in their spare time. Common to these programmers is that they learn programming from the top down.

Most of the time, this is perfectly fine, but on the topic of asynchronous programming in general, and Rust in particular, there is a clear advantage to learning about the topic from first principles, and this book aims to provide a means to do just that.

Asynchronous programming is a way to write programs where you divide your program into tasks that can be stopped and resumed at specific points. This, in turn, allows a language runtime, or a library, to drive and schedule these tasks so their progress interleaves.

Asynchronous programming will, by its very nature, affect the entire program flow, and it’s very invasive. It rewrites, reorders, and schedules the program you write in a way that’s not always obvious to you as a programmer.

Most programming languages try to make asynchronous programming so easy that you don’t really have to understand how it works just to be productive in it.

You can get quite productive writing asynchronous Rust without really knowing how it works as well, but Rust is more explicit and surfaces more complexity to the programmer than most other languages. You will have a much easier time handling this complexity if you get a deep understanding of asynchronous programming in general and what really happens when you write asynchronous Rust.

Another huge upside is that learning from first principles results in knowledge that is applicable way beyond Rust, and it will, in turn, make it easier to pick up asynchronous programming in other languages as well. I would even go so far as to say that most of this knowledge will be useful even in your day-to-day programming. At least, that’s how it’s been for me.

I want this book to feel like you’re joining me on a journey, where we build our knowledge topic by topic and learn by creating examples and experiments along the way. I don’t want this book to feel like a lecturer simply telling you how everything works.

This book is created for people who are curious by nature, the kind of programmers who want to understand the systems they use, and who like creating small and big experiments as a way to explore and learn.

Who this book is for

This book is for developers with some prior programming experience who want to learn asynchronous programming from the ground up so they can be proficient in async Rust and be able to participate in technical discussions on the subject. The book is perfect for those who like writing working examples they can pick apart, expand, and experiment with.
There are two kinds of personas that I feel this book is especially relevant to:

• Developers coming from higher-level languages with a garbage collector, interpreter, or runtime, such as C#, Java, JavaScript, Python, Ruby, Swift, or Go. Programmers who have extensive experience with asynchronous programming in any of these languages but want to learn it from the ground up and programmers with no experience with asynchronous programming should both find this book equally useful.
• Developers with experience in languages such as C or C++ that have limited experience with asynchronous programming.

What this book covers

Chapter 1, Concurrency and Asynchronous Programming: A Detailed Overview, provides a short history leading up to the type of asynchronous programming we use today. We give several important definitions and provide a mental model that explains what kind of problems asynchronous programming really solves, and how concurrency differs from parallelism. We also cover the importance of choosing the correct reference frame when discussing asynchronous program flow, and we go through several important and fundamental concepts about CPUs, operating systems, hardware, interrupts, and I/O.

Chapter 2, How Programming Languages Model Asynchronous Program Flow, narrows the scope from the previous chapter and focuses on the different ways programming languages deal with asynchronous programming. It starts by giving several important definitions before explaining stackful and stackless coroutines, OS threads, green threads, fibers, callbacks, promises, futures, and async/await.

Chapter 3, Understanding OS-Backed Event Queues, System Calls, and Cross-Platform Abstractions, explains what epoll, kqueue, and IOCP are and how they differ. It prepares us for the next chapters by giving an introduction to syscalls, FFI, and cross-platform abstractions.

Chapter 4, Create Your Own Event Queue, is the chapter where you create your own event queue that mimics the API of mio (the popular Rust library that underpins much of the current async ecosystem). The example will center around epoll and go into quite a bit of detail on how it works.

Chapter 5, Creating Our Own Fibers, walks through an example where we create our own kind of stackful coroutines called fibers. They’re the same kind of green threads that Go uses and show one of the most widespread and popular alternatives to the type of abstraction Rust uses with futures and async/await today. Rust used this kind of abstraction in its early days before it reached 1.0, so it’s also a part of Rust’s history. This chapter will also cover quite a few general programming concepts, such as stacks, assembly, Application Binary Interfaces (ABIs), and instruction set architecture (ISAs), that are useful beyond the context of asynchronous programming as well.

Chapter 6, Futures in Rust, gives a short introduction and overview of futures, runtimes, and asynchronous programming in Rust.

Chapter 7, Coroutines and async/await, is a chapter where you write your own coroutines that are simplified versions of the ones created by async/await in Rust today. We’ll write a few of them by hand and introduce a new syntax that allows us to programmatically rewrite what look like regular functions into the coroutines we wrote by hand.

Chapter 8, Runtimes, Wakers, and the Reactor-Executor Pattern, introduces runtimes and runtime design. By iterating on the example we created in Chapter 7, we’ll create a runtime for our coroutines that we’ll gradually improve. We’ll also do some experiments with our runtime once it’s done to better understand how it works.

Chapter 9, Coroutines, Self-Referential Structs, and Pinning, is the chapter where we introduce self-referential structs and pinning in Rust. By improving our coroutines further, we’ll experience first-hand why we need something such as Pin, and how it helps us solve the problems we encounter.

Chapter 10, Create Your Own Runtime, is the chapter where we finally put all the pieces together. We’ll improve the same example from the previous chapters further so we can run Rust futures, which will allow us to use the full power of async/await and asynchronous Rust. We’ll also do a few experiments that show some of the difficulties with asynchronous Rust and how we can best solve them.

To get the most out of this book

You should have some prior programming experience and, preferably, some knowledge about Rust. Reading the free, and excellent, introductory book The Rust Programming Language (https://doc.rust-lang.org/book/) should give you more than enough knowledge about Rust to follow along since any advanced topics will be explained step by step.

The ideal way to read this book is to have the book and a code editor open side by side. You should also have the accompanying repository available so you can refer to that if you encounter any issues.

Software/hardware covered in the book	Operating system requirements
Rust (version 1.51 or later)	Windows, macOS, or Linux

You need Rust installed. If you haven’t already, follow the instructions here: https://www.rust-lang.org/tools/install.

Some examples will require you to use Windows Subsystem for Linux (WSL) on Windows. If you’re following along on a Windows machine, I recommend that you enable WSL (https://learn.microsoft.com/en-us/windows/wsl/install) now and install Rust by following the instructions for installing Rust on WSL here: https://www.rust-lang.org/tools/install.

If you are using the digital version of this book, we advise you to type the code yourself or access the code from the book’s GitHub repository (a link is available in the next section). Doing so will help you avoid any potential errors related to the copying and pasting of code.

The accompanying repository is organized in the following fashion:

• Code that belongs to a specific chapter is in that chapter’s folder (e.g., ch01).
• Each example is organized as a separate crate.
• The letters in front of the example names indicate in what order the different examples are presented in the book. For example, the a-runtime example comes before the b-reactor-executor example. This way, they will be ordered chronologically (at least by default on most systems).
• Some examples have a version postfixed with -bonus. These versions will be mentioned in the book text and often contain a specific variant of the example that might be interesting to check out but is not important to the topic at hand.

Download the example code files

You can download the example code files for this book from GitHub at https://github.com/PacktPublishing/Asynchronous-Programming-in-Rust. If there’s an update to the code, it will be updated in the GitHub repository.

We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!

Conventions used

There are a number of text conventions used throughout this book.

Code in text: Indicates code words in text, database table names, folder names, filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. Here is an example: “So, now we have created our own async runtime that uses Rust’s Futures, Waker, Context, and async/await.”

A block of code is set as follows:

pub trait Future {
    type Output;
    fn poll(&mut self) -> PollState<Self::Output>;
}

When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:

struct Coroutine0 {
    stack: Stack0,
    state: State0,
}

Any command-line input or output is written as follows:

$ cargo run

Tips or important notes

Appear like this.

Get in touch

Feedback from our readers is always welcome.

General feedback: If you have questions about any aspect of this book, email us at [e-mail] and mention the book title in the subject of your message.

Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book, we would be grateful if you would report this to us. Please visit www.packtpub.com/support/errata and fill in the form.

Piracy: If you come across any illegal copies of our works in any form on the internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [e-mail] with a link to the material.

If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.

Share your thoughts

Once you’ve read Asynchronous Programming in Rust, we’d love to hear your thoughts! Please click here to go straight to the Amazon review page for this book and share your feedback.

Your review is important to us and the tech community and will help us make sure we’re delivering excellent quality content.

Download a free PDF copy of this book

Thanks for purchasing this book!

Do you like to read on the go but are unable to carry your print books everywhere?

Is your eBook purchase not compatible with the device of your choice?

Don’t worry, now with every Packt book you get a DRM-free PDF version of that book at no cost.

Read anywhere, any place, on any device. Search, copy, and paste code from your favorite technical books directly into your application.

The perks don’t stop there, you can get exclusive access to discounts, newsletters, and great free content in your inbox daily

Follow these simple steps to get the benefits:

1. Scan the QR code or visit the link below

https://packt.link/free-ebook/9781805128137

2. Submit your proof of purchase

3. That’s it! We’ll send your free PDF and other benefits to your email directly