Understanding OS-Backed Event Queues, System Calls, and Cross-Platform Abstractions

In this chapter, we’ll take a look at how an OS-backed event queue works and how three different operating systems handle this task in different ways. The reason for going through this is that most async runtimes I know of use OS-backed event queues such as this as a fundamental part of achieving high-performance I/O. You’ll most likely hear references to these frequently when reading about how async code really works.

Event queues based on the technology we discuss in this chapter is used in many popular libraries like:

• mio (https://github.com/tokio-rs/mio), a key part of popular runtimes like Tokio
• polling (https://github.com/smol-rs/polling), the event queue used in Smol and async-std
• libuv (https://libuv.org/), the library used to create the event queue used in Node.js (a JavaScript runtime) and the Julia programming language
• C# for its asynchronous network calls...

Technical requirements

This chapter doesn’t require you to set up anything new, but since we’re writing some low-level code for three different platforms, you need access to these platforms if you want to run all the examples.

The best way to follow along is to open the accompanying repository on your computer and navigate to the ch03 folder.

This chapter is a little special since we build some basic understanding from the ground up, which means some of it is quite low-level and requires a specific operating system and CPU family to run. Don’t worry; I’ve chosen the most used and popular CPU, so this shouldn’t be a problem, but it is something you need to be aware of.

The machine must use a CPU using the x86-64 instruction set on Windows and Linux. Intel and AMD desktop CPUs use this architecture, but if you run Linux (or WSL) on a machine using an ARM processor you might encounter issues with some of the examples using inline assembly. On...

Why use an OS-backed event queue?

You already know by now that we need to cooperate closely with the OS to make I/O operations as efficient as possible. Operating systems such as Linux, macOS, and Windows provide several ways of performing I/O, both blocking and non-blocking.

I/O operations need to go through the operating system since they are dependent on resources that our operating system abstracts over. This can be the disk drive, the network card, or other peripherals. Especially in the case of network calls, we’re not only dependent on our own hardware, but we also depend on resources that might reside far away from our own, causing a significant delay.

In the previous chapter, we covered different ways to handle asynchronous operations when programming, and while they’re all different, they all have one thing in common: they need control over when and if they should yield to the OS scheduler when making a syscall.

In practice, this means that syscalls...

Readiness-based event queues

epoll and kqueue are known as readiness-based event queues, which means they let you know when an action is ready to be performed. An example of this is a socket that is ready to be read from.

To give an idea about how this works in practice, we can take a look at what happens when we read data from a socket using epoll/kqueue:

1. We create an event queue by calling the syscall epoll_create or kqueue.
2. We ask the OS for a file descriptor representing a network socket.
3. Through another syscall, we register an interest in Read events on this socket. It’s important that we also inform the OS that we’ll be expecting to receive a notification when the event is ready in the event queue we created in step 1.
4. Next, we call epoll_wait or kevent to wait for an event. This will block (suspend) the thread it’s called on.
5. When the event is ready, our thread is unblocked (resumed) and we return from our wait call with data about...

Completion-based event queues

IOCP stands for input/output completion port. This is a completion-based event queue. This type of queue notifies you when events are completed. An example of this is when data has been read into a buffer.

The following is a basic breakdown of what happens in this type of event queue:

1. We create an event queue by calling the syscall CreateIoCompletionPort.
2. We create a buffer and ask the OS to give us a handle to a socket.
3. We register an interest in Read events on this socket with another syscall, but this time we also pass in the buffer we created in (step 2) , which the data will be read to.
4. Next, we call GetQueuedCompletionStatusEx, which will block until an event has been completed.
5. Our thread is unblocked and our buffer is now filled with the data we’re interested in.

Figure 3.2 – A simplified view of the IOCP flow

epoll, kqueue, and IOCP

epoll is the Linux way of implementing an event queue. In terms of functionality, it has a lot in common with kqueue. The advantage of using epoll over other similar methods on Linux, such as select or poll, is that epoll was designed to work very efficiently with a large number of events.

kqueue is the macOS way of implementing an event queue (which originated from BSD) in operating systems such as FreeBSD and OpenBSD. In terms of high-level functionality, it’s similar to epoll in concept but different in actual use.

IOCP is the way Windows handle this type of event queue. In Windows, a completion port will let you know when an event has been completed. Now, this might sound like a minor difference, but it’s not. This is especially apparent when you want to write a library since abstracting over both means you’ll either have to model IOCP as readiness-based or model epoll/kqueue as completion-based.

Lending out a buffer to the...

Cross-platform event queues

When creating a cross-platform event queue, you have to deal with the fact that you have to create one unified API that’s the same whether it’s used on Windows (IOCP), macOS (kqueue), or Linux (epoll). The most obvious difference is that IOCP is completion-based while kqueue and epoll are readiness-based.

This fundamental difference means that you have to make a choice:

• You can create an abstraction that treats kqueue and epoll as completion-based queues, or
• You can create an abstraction that treats IOCP as a readiness-based queue

From my personal experience, it’s a lot easier to create an abstraction that mimics a completion-based queue and handle the fact that kqueue and epoll are readiness-based behind the scenes than the other way around. The use of wepoll, as I alluded to earlier, is one way of creating a readiness-based queue on Windows. It will simplify creating such an API greatly, but we’ll leave...

System calls, FFI, and cross-platform abstractions

We’ll implement a very basic syscall for the three architectures: BSD/macOS, Linux, and Windows. We’ll also see how this is implemented in three levels of abstraction.

The syscall we’ll implement is the one used when we write something to the standard output (stdout) since that is such a common operation and it’s interesting to see how it really works.

We’ll start off by looking at the lowest level of abstraction we can use to make system calls and build our understanding of them from the ground up.

The lowest level of abstraction

The lowest level of abstraction is to write what is often referred to as a “raw” syscall. A raw syscall is one that bypasses the OS-provided library for making syscalls and instead relies on the OS having a stable syscall ABI. A stable syscall ABI means it guarantees that if you put the right data in certain registers and call a specific CPU instruction...

Summary

In this chapter, we went through what OS-backed event queues are and gave a high-level overview of how they work. We also went through the defining characteristics of epoll, kqueue, and IOCP and focused on how they differ from each other.

In the last half of this chapter, we introduced some examples of syscalls. We discussed raw syscalls, and “normal” syscalls so that you know what they are and have seen examples of both. We also took the opportunity to talk about abstraction levels and the advantages of relying on good abstractions when they’re available to us.

As a part of making system calls, you also got an introduction to Rusts FFI.

Finally, we created a cross-platform abstraction. You also saw some of the challenges that come with creating a unifying API that works across several operating systems.

The next chapter will walk you through an example using epoll to create a simple event queue, so you get to see exactly how this works in practice...