Kernel Internals Essentials – Processes and Threads

With the previous chapter, you’re now in a good position to understand and write simple kernel modules. In this chapter, we begin our exploration of Linux kernel internals, a vast and complex topic. In this book, we do not intend to delve very deep into the details of kernel and memory internals. At the same time, I would like to provide sufficient and requisite background knowledge for a budding kernel module author or device driver developer like you to successfully tackle the key topics necessary to understand kernel architecture in terms of how processes, threads, and their stacks are managed. Armed with this knowledge, you’ll be able to better understand the coming chapters on correctly and efficiently managing dynamic kernel memory. As a side benefit, you will find yourself becoming more proficient at debugging both user and kernel space code.

I have divided the discussion on essential kernel internals...

Technical requirements

I assume that you have gone through Online Chapter, Kernel Workspace Setup, and have appropriately prepared a guest Virtual Machine (VM) running Ubuntu 22.04 LTS (or a later stable release) and installed all the required packages. If not, I recommend you to do this first.

Also, if you haven’t yet, do clone this book’s GitHub repository for the code (found here: https://github.com/PacktPublishing/Linux-Kernel-Programming_2E); let’s work on it in a hands-on fashion.

I assume that you are familiar with basic virtual memory concepts, the user-mode process Virtual Address Space (VAS) layout of segments (mappings), the stack, and so on. Nevertheless, we devote a few pages to explaining these basics (in the Understanding the basics of the process VAS section that soon follows).

Understanding process and interrupt contexts

In Chapter 4, Writing Your First Kernel Module – Part 1, we presented a brief section entitled Understanding kernel architecture – part 1 (if you haven’t read it yet, I suggest you do so before continuing). We will now expand on this discussion.

First off, modern processors execute code at different levels of privilege. For example, the x86-based ones offer four levels (or rings) of privilege, with Ring 0 being the most privileged and Ring 3 being the least. Similarly, the ARM-32 (AArch32) has seven execution modes, six of which are privileged. ARM64 (AArch64) uses the notion of exception levels (EL0 to EL3, with EL0 being the least and EL3 being the most privileged). Realistically though, and a key point: all modern OSs employ just two of the available CPU privilege levels – a privileged level and an unprivileged one at which code executes; we refer to them as kernel and user mode, respectively.

It&...

Understanding the basics of the process Virtual Address Space (VAS)

A fundamental ‘rule’ of virtual memory is this: all potentially addressable memory is in a box; that is, it’s sandboxed. We think of this ‘box’ as the process image or the process VAS. Looking ‘outside’ the box is impossible.

Here, we provide only a quick overview of the process user VAS (which should be sufficient). For more details, please refer to my earlier book Hands-On System Programming with Linux.

The user VAS is divided into homogeneous memory regions called segments or, more technically, mappings (as they’re internally constructed via the mmap() system call). Figure 6.2 shows the minimal mappings (segments) that every single Linux (user space) process will have:

Figure 6.2: Linux user space process VAS

Let’s go over a quick breakdown of these segments or mappings (from the bottom up):

• Text segment:...

Organizing processes, threads, and their stacks – user and kernel space

The traditional UNIX process model – Everything is a process; if it’s not a process, it’s a file – has a lot going for it. The very fact that it is still the model followed by operating systems after a span of over five decades amply validates this. Of course, nowadays, the thread is considered the atomic execution context; a thread is an execution path within a process. Threads share all process resources, including the user VAS, open files, signal dispositions, IPC objects, credentials, paging tables, and so on, except for the stack. Every thread has its own private stack region (this makes perfect sense; if not, how could threads truly run in parallel, as it’s the stack that holds execution context).

The other reason we focus on the thread and not the process is made clearer in Chapter 10, The CPU Scheduler – Part 1. For now, we shall just say this: the thread...

Understanding and accessing the kernel task structure

As you have learned by now, every single user and kernel space thread is internally represented within the Linux kernel by a metadata structure containing all its attributes – the task structure. The task structure is represented within the kernel code here: include/linux/sched.h:struct task_struct.

To view any version of the kernel code online, a superb (searchable) system is in place here: https://elixir.bootlin.com/linux/latest/source. For the 6.1.25 LTS kernel release, for example, here’s the task structure definition: https://elixir.bootlin.com/linux/v6.1.25/source/include/linux/sched.h#L737.

It’s often, unfortunately, referred to as the “process descriptor,” causing no end of confusion! Thankfully, the phrase task structure is so much better; it represents a runnable task – in effect, a thread.

So, there we have it: in the Linux design, every process consists...

Working with the task structure via ‘current’

Here, we will write a simple kernel module to show a few members of the task structure. Further, I want you to think about this: who exactly is running the code of this (or any) kernel module’s (or the kernel’s) init and cleanup code paths? From what we’ve learned, it’s not the kernel; as stated before, there’s no grand overall “kernel” process as such… Then, who’s running it?

The answer should be clear in a monolithic kernel such as the Linux OS: when a user space process (or thread) issues a system call, it switches to kernel mode and runs kernel (or module) code in process context. So, yes, it will be a process (or thread). Which one? The code we write will reveal the process context that our module’s init and cleanup code paths run in. (We discuss this very point in more detail in the upcoming Seeing that the Linux OS is monolithic section.) To...

Iterating over the kernel’s task lists

As mentioned earlier, all the task structures are organized in kernel memory in a linked list called the task list (allowing them to be iterated over). The list data structure has evolved to become the very commonly used circular doubly linked list. In fact, the core kernel code to work with these lists has been factored out into a header called list.h; it’s well-known and expected to be used for any list-based work.

The include/linux/types.h:list_head data structure forms the essential doubly linked circular list; as expected, it consists of two pointers, one to the prev member on the list and one to the next member:

struct list_head {
        struct list_head *next, *prev;
};

You can easily iterate over various lists concerned with tasks via conveniently provided macros in the include/linux/sched/signal.h header file for versions >= 4.11; note that for kernels 4.10 and older, the macros are in include/linux/sched...

Summary

In this chapter, we covered the key aspects of kernel internals that will help you as a kernel module or device driver author to better and more deeply understand the internal workings of the OS. We examined, in some detail, the organization of and relationships between the process and its threads and stacks (in both user and kernel space). We examined the kernel task_struct metadata structure and learned how to iterate over the task list in different ways via our custom written kernel modules.

Though it may not be obvious, the fact is that understanding these kernel internal details is a necessary and required step in your journey to becoming a seasoned kernel (and/or device driver) developer. The content of this chapter will help you debug many system programming scenarios and it lays the foundation for our deeper exploration into the Linux kernel, particularly that of memory management.

The next chapter and the couple that follow it are critical indeed: we’...

Questions

As we conclude, here is a list of questions for you to test your knowledge regarding this chapter’s material: https://github.com/PacktPublishing/Linux-Kernel-Programming_2E/blob/main/questions/ch6_qs_assignments.txt. You will find some of the questions answered in the book’s GitHub repo: https://github.com/PacktPublishing/Linux-Kernel-Programming_2E/tree/main/solutions_to_assgn.

Further reading

To help you delve deeper into the subject with useful materials, we provide a rather detailed list of online references and links (and at times, even books) in a Further reading document in this book’s GitHub repository. The Further reading document is available here: https://github.com/PacktPublishing/Linux-Kernel-Programming_2E/blob/main/Further_Reading.md.

Leave a review!

Enjoying this book? Help readers like you by leaving an Amazon review. Scan the QR code below for a 20% discount code.

*Limited Offer