Memory Management Internals – Essentials

Kernel internals, especially regarding memory management, is a vast and complex topic. In this book, I do not intend to delve into the deep, gory details of kernel memory internals. At the same time, I would like to provide sufficient – and required – background knowledge for a budding kernel or device driver developer like you to successfully tackle this key topic.

Accordingly, this chapter will help you understand, to sufficient depth, the internals of how memory management is performed on the Linux OS; this includes delving into the Virtual Memory (VM) split, examining both the user-mode and kernel VAS of the process to a good level of depth, and covering the basics of how the kernel manages physical memory. In effect, you will come to understand the memory maps – both virtual and (to some extent) physical – of the process and the system.

This background knowledge will go a long way in helping you...

Technical requirements

I assume that you have gone through Online Chapter, Kernel Workspace Setup, and have appropriately prepared a guest VM (or native system) running Ubuntu 22.04 LTS (or a later stable release) and installed all the required packages. If not, I recommend you do this first. To get the most out of this book, I strongly recommend you first set up the workspace environment, including cloning this book’s GitHub repository for the code (https://github.com/PacktPublishing/Linux-Kernel-Programming_2E), and work on it in a hands-on fashion.

I assume that you are familiar with the basic virtual memory concepts, the user-mode process Virtual Address Space (VAS) layout of segments, user and kernel-mode stacks, the task structure, and so on. If you’re unsure on this footing, I strongly suggest you read the preceding chapter first.

Understanding the VM split

In this chapter, we will broadly look at how the Linux kernel manages memory in two ways:

• The virtual memory-based approach, where memory is virtualized (the usual case)
• A view of how the kernel organizes physical memory (RAM pages)

First, let’s begin with the virtual memory view and then discuss physical memory organization later in the chapter.

As we saw in Chapter 6, Kernel Internals Essentials – Processes and Threads, in the Understanding the basics of the process Virtual Address Space (VAS) section, a key property of the process VAS is that it is completely self-contained, a sandbox. You cannot look outside the box. In that chapter, in Figure 6.2, we saw that the process VAS ranges from virtual address 0x0 to what we simply termed as the “high address.” What is the actual value of this “high” address? It’s the highest extent of the VAS and thus depends on the number of bits...

Examining the process VAS

We have already covered the layout – the segments or mappings – that every process’s VAS is made up of (see the Understanding the basics of the process Virtual Address Space (VAS) section in Chapter 6, Kernel Internals Essentials – Processes and Threads). We learned that the process VAS consists of various mappings or segments; among them are text (code), data segments, library mappings, and at least one stack. Here, we will expand greatly on that discussion.

Being able to dive deep into the kernel and see various runtime values is an important skill for a developer like you (as well as for the app user, QA, sysadmin, DevOps folks, and so on). The Linux kernel provides us with an amazing interface to do precisely this – it’s, you guessed it, the proc filesystem (procfs).

This pseudo filesystem is always present on Linux (at least it should be) and is mounted under /proc by default. The procfs system has two...

Examining the kernel VAS

As we have talked about in the preceding chapter, and as seen in Figure 7.7, it’s critical to understand that all processes have their own unique user VAS but share the kernel space – what we call the kernel segment or kernel VAS. Let’s begin this section by starting to examine some common (arch-independent) regions of the kernel VAS.

The kernel VAS’s memory layout is very arch (CPU)-dependent. Nevertheless, all architectures share some commonalities. The following basic diagram represents both the user VAS and the kernel VAS (in a horizontally tiled format), as seen on a typical x86_32 (or IA-32) with a 3:1 (GB) VM split:

Figure 7.12: User and kernel VASs on an x86_32 with a 3:1 (GB) VM split with a focus on the lowmem region; this figure is deliberately simplistic

Let’s go over each region of the process VAS (from left to right, as seen in Figure 7.12):

• The user mode VAS: This is the user VAS...

Randomizing the memory layout – KASLR

In infosec circles, it’s a well-known fact that, with proc filesystem (procfs) and various powerful “hacking” tools at their disposal (heard of Kali Linux?), a malicious user, knowing in advance the precise location (virtual addresses) of various functions and/or globals within a process’s VAS, could devise an attack to exploit and ultimately compromise a given system. (Why, even knowing the precise location of one well-known function or global in a given kernel could lead to an attack vector!) Thus, for security, to make it near impossible (or at least difficult) for attackers to rely on “known” virtual addresses, user space, as well as kernel space, supports Address Space Layout Randomization (ASLR) and Kernel ASLR (KASLR) techniques (often pronounced Ass-ler/Kass-ler).

The keyword here is randomization: this feature, when enabled, changes the location of portions of the process (and kernel...

Understanding physical memory organization

Now that we have examined the virtual memory view for both user and kernel VASs in quite some detail, let’s turn to the topic of physical memory organization on the Linux OS.

Physical RAM organization

The Linux kernel, at boot, organizes and partitions physical RAM into a tree-like hierarchy consisting of nodes, zones, and page frames (page frames are physical pages of RAM) (see Figure 7.22 and Figure 7.23). Do note that further organization via physical memory models is also performed at early boot and is a related discussion; we will throw some light on this in the An introduction to physical memory models section.

Nodes are divided into zones, and zones consist of page frames. It’s essentially a tree-like hierarchy, simplistically and conceptually depicted as a three-level tree-like hierarchy like this:

• Node(s) ← Level 1
  • Zone(s) ← Level 2
    • Page frames ← Level...

Summary

In this chapter, we delved – in quite some depth – into the big topic of kernel memory management in a level of detail sufficient for a kernel module or device driver author like you; also, there’s more to come! A key piece of the puzzle – the VM split and how it’s achieved on various architectures running the Linux OS – served as a starting point.

We then moved into a deep examination of both regions of this split: first, user space (the user mode process VAS) and then the kernel VAS (or kernel segment). Here, we covered many details and tools/utilities on how to examine it (including via the quite powerful procmap utility). We built a demo kernel module that can literally generate a pretty complete memory map of the kernel and the calling process. User and kernel memory layout randomization technology ([K]ASLR) was also briefly discussed. We closed the chapter by looking at the physical organization of RAM within the Linux OS...

Questions

As we conclude, here is a list of questions for you to test your knowledge regarding this chapter’s material: https://github.com/PacktPublishing/Linux-Kernel-Programming_2E/blob/main/questions/ch7_qs_assignments.txt. You will find some of the questions answered in the book’s GitHub repo: https://github.com/PacktPublishing/Linux-Kernel-Programming_2E/tree/master/solutions_to_assgn.

Further reading

To help you delve deeper into the subject with useful materials, we provide a rather detailed list of online references and links (and at times, even books) in a Further reading document in this book’s GitHub repository. The Further reading document is available here: https://github.com/PacktPublishing/Linux-Kernel-Programming_2E/blob/master/Further_Reading.md.

Learn more on Discord

To join the Discord community for this book – where you can share feedback, ask questions to the author, and learn about new releases – follow the QR code below:

https://packt.link/SecNet