Kernel Memory Management

On Linux systems, every memory address is virtual. They do not point to any address in the RAM directly. Whenever you access a memory location, a translation mechanism is performed in order to match the corresponding physical memory.

Let's start with a short story to introduce the virtual memory concept. Given a hotel, there can be a phone in each room, each with a private number. Any installed phone, of course, belongs to the hotel. None of them can be joined directly from outside the hotel.

If you need to contact an occupant of a room, let's say your friend, he must give you the hotel's switchboard number and the room number in which he is staying. Once you call the switchboard and give the room number of the occupant you need to talk to, the receptionist redirects your call to the private phone in the room. Only the receptionist and the...

System memory layout - kernel space and user space

Throughout this chapter, terms such as kernel space and user space will refer to their virtual address space. On Linux systems, each process owns a virtual address space. It is a kind of memory sandbox during the life of the process. That address space is 4 GB in size on 32-bit systems (even on a system with physical memory less than 4 GB). For each process, that 4 GB address space is split into two parts:

• User space virtual addresses
• Kernel space virtual addresses

The way the split is done depends on a special kernel configuration option, CONFIG_PAGE_OFFSET, which defines where the kernel addresses section starts in a process address space. The common value is 0xC0000000 by default on 32-bit systems, but this may be changed, as is the case for i.MX6 family processors from NXP, which use 0x80000000. In the whole...

Address translation and MMU

Virtual memory is a concept, an illusion given to a process so it thinks it has large and almost infinite memory, and sometimes more than the system really has. It is up to the CPU to make the conversion from virtual to physical address every time you access a memory location. That mechanism is called address translation, and is performed by the Memory Management Unit (MMU), which is a part of the CPU.

MMU protects memory from unauthorized access. Given a process, any page that needs to be accessed must exist in one of the process VMAs, and thus must live in the process page table (every process has its own).

Memory is organized by chunks of fixed size named pages for virtual memory and frames for physical memory, sized 4 KB in our case. Anyway, you do not need to guess the page size of the system you write the driver for. It is defined and accessible...

Memory allocation mechanism

Let's look at the following diagram, showing us the different memory allocators that exist on a Linux-based system, and discuss it later. (inspired by http://free-electrons.com/doc/training/linux-kernel/linux-kernel-slides.pdf):

Overview of kernel memory allocator

There is an allocation mechanism to satisfy any kind of memory request. Depending on what you need memory for, you can choose the one closest to your goal. The main allocator is the Page Allocator, which only works with pages (a page being the smallest memory unit it can deliver). Then comes the SLAB Allocator which is built on top of the page allocator, getting pages from it and returning smaller memory entities (by mean of slabs and caches). This is the allocator on which the kmalloc Allocator relies.

...

Work with I/O memory to talk with hardware

Apart from performing data RAM-oriented operations, you can perform I/O memory transactions to talk with the hardware. When it comes to the access device's register, the kernel offers two possibilities depending on the system architecture:

• Through the I/O ports: This is also called Port Input Output (PIO). Registers are accessible through a dedicated bus, and specific instructions (in and out, in assembler generally) are needed to access those registers. This is the case on x86 architectures.
• Memory Mapped Input Output (MMIO): This is the most common and most used method. The device's registers are mapped to memory. Simply read and write to a particular address to write to the registers of the device. This is the case on ARM architectures.

...

Memory (re)mapping

Kernel memory sometimes needs to be remapped, either from kernel to user space, or from kernel to kernel space. The common use case is remapping the kernel memory to the user space, but there are other cases when you need to access high memory, for example.

kmap

The Linux kernel permanently maps 896 MB of its address space to the lower 896 MB of the physical memory (low memory). On a 4 GB system, there is only 128 MB left to the kernel to map the remaining 3.2 GB of physical memory (high memory). Low memory is directly addressable by the kernel because of the permanent and one-to-one mapping. When it comes to high memory (memory above 896 MB), the kernel has to map the requested region of high memory into...

Linux caching system

Caching is the process by which frequently accessed or newly written data is fetched from or written to a small and faster memory, called a cache.

Dirty memory is data-backed (for example, file-backed) memory whose content has been modified (typically in a cache) but not written back to the disk yet. The cached version of the data is newer than the on-disk version, meaning that both versions are out of sync. The mechanism by which cached data is written back on the disk (back store) is called writeback. We will eventually update the on-disk version, bringing the two in sync. Clean memory is file-backed memory in which the contents are in sync with the disk.

Linux delays write operations in order to speed up the read process, and reduces disk wear leveling by writing data only when necessary. A typical example is the dd command. Its complete execution does...

Device-managed resources – Devres

Devres is a kernel facility helping the developer by automatically freeing the allocated resource in a driver. It simplifies errors handling in init/probe/open functions. With devres, each resource allocator has its managed version that will take care of resource release and freeing for you.

This section heavily relies on the Documentation/driver-model/devres.txt file in the kernel source tree, which deals with the devres API and lists supported functions along with their descriptions.

The memory allocated with resource-managed functions is associated with the device. devres consists of a linked list of arbitrarily sized memory areas associated with a struct device. Each devers resource allocator inserts the allocated resource in the list. The resource remains available until it is manually freed by the code, when the device is detached...

Summary

This chapter is one of the most important chapters. It demystifies memory management and allocation (how and where) in the kernel. Every memory aspect is discussed and detailed, as well as dvres is also explained. The caching mechanism is briefly discussed in order to give an overview of what goes on under the hood during I/O operations. It is a strong base from which to introduce and understand the next chapter, which deals with DMA.