Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Save more on your purchases now! discount-offer-chevron-icon
Savings automatically calculated. No voucher code required.
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Linux Kernel Programming
Linux Kernel Programming

Linux Kernel Programming: A comprehensive guide to kernel internals, writing kernel modules, and kernel synchronization

Arrow left icon
Profile Icon Kaiwan N. Billimoria
Arrow right icon
€41.99
Paperback Mar 2021 754 pages 1st Edition
eBook
€22.99 €32.99
Paperback
€41.99
Subscription
Free Trial
Renews at €18.99p/m
Arrow left icon
Profile Icon Kaiwan N. Billimoria
Arrow right icon
€41.99
Paperback Mar 2021 754 pages 1st Edition
eBook
€22.99 €32.99
Paperback
€41.99
Subscription
Free Trial
Renews at €18.99p/m
eBook
€22.99 €32.99
Paperback
€41.99
Subscription
Free Trial
Renews at €18.99p/m

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
Table of content icon View table of contents Preview book icon Preview Book

Linux Kernel Programming

Kernel Workspace Setup

Hello, and welcome to this book on learning Linux kernel development. To get the most out of this book, it is very important that you first set up the workspace environment that we will be using throughout the book. This chapter will teach you exactly how to do this and get started.

We will install a recent Linux distribution, preferably as a Virtual Machine (VM), and set it up to include all the required software packages. We will also clone this book's code repository on GitHub, and learn about a few useful projects that will help along this journey.

The best way to learn something is to do so empirically – not taking anyone's word on anything at all, but trying it out and experiencing it for yourself. Hence, this book gives you many hands-on experiments and kernel code examples that you can and indeed must try out yourself; this will greatly aid in your making real progress and deeply learning and understanding various aspects of Linux kernel and driver development. So, let's begin!

This chapter will take us through the following topics, which will help us set up our environment:

  • Running Linux as a guest VM
  • Setting up the software – distribution and packages
  • A few additional useful projects

Technical requirements

You will need a modern desktop PC or laptop. Ubuntu Desktop specifies the following as "recommended system requirements" for the installation and usage of the distribution:

  • A 2 GHz dual core processor or better.
  • RAM:
    • Running on physical host: 2 GB or more system memory (more will certainly help).
    • Running as a guest VM: The host system should have at least 4 GB RAM (the more the better and the smoother the experience).
  • 25 GB of free hard drive space (I suggest more, at least double this).
  • Either a DVD drive or a USB port for the installer media (not required when setting up Ubuntu as a guest VM).
  • Internet access is definitely helpful and required at times.

As performing tasks such as building a Linux kernel from source is a very memory- and CPU-intensive process, I highly recommend that you try it out on a powerful Linux system with plenty of RAM and disk space to spare as well. It should be pretty obvious – the more RAM and CPU power the host system has, the better!

Like any seasoned kernel contributor, I would say that working on a native Linux system is best. However, for the purposes of this book, we cannot assume that you will always have a dedicated native Linux box available to you. So, we will assume that you are working on a Linux guest. Working within a guest VM also adds an additional layer of isolation and thus safety. 

Cloning our code repository: The complete source code for this book is freely available on GitHub at https://github.com/PacktPublishing/Linux-Kernel-Programming. You can clone and work on it by cloning the git tree, like so:

git clone https://github.com/PacktPublishing/Linux-Kernel-Programming.git

The source code is organized chapter-wise. Each chapter is represented as a directory – for example, ch1/ has the source code for this chapter. The root of the source tree has some code that is common to all chapters, such as the source files convenient.h, klib_llkd.c, as well as others.

For efficient code browsing, I would strongly recommend that you always index the code base with ctags(1) and/or cscope(1). For example, to set up the ctags index, just cd to the root of the source tree and type ctags -R .

Unless noted otherwise, the code output we show in the book is the output as seen on an x86-64 Ubuntu 18.04.3 LTS guest VM (running under Oracle VirtualBox 6.1). You should realize that due to (usually minor) distribution – and even within the same distributions but differing versions – differences, the output shown here may not perfectly match what you see on your Linux system.

Running Linux as a guest VM

As discussed previously, a practical and convenient alternative to using a native Linux system is to install and use the Linux distribution as a guest OS on a VM. It's key that you install a recent Linux distribution, preferably as a VM to be safe and avoid unpleasant data loss or other surprises. The fact is when working at the level of the kernel, abruptly crashing the system (and the data loss risks that arise thereof) is actually a commonplace occurrence. I recommend using Oracle VirtualBox 6.x (or the latest stable version) or other virtualization software, such as VMware Workstation.

Both of these are freely available. It's just that the code for this book has been tested on VirtualBox 6.1. Oracle VirtualBox is considered Open Source Software (OSS) and is licensed under the GPL v2 (the same as the Linux kernel). You can download it from https://www.virtualbox.org/wiki/Downloads. Its documentation can be found here: https://www.virtualbox.org/wiki/End-user_documentation.

The host system should be either MS Windows 10 or later (of course, even Windows 7 will work), a recent Linux distribution (for example, Ubuntu or Fedora), or macOS. So, let's get started by installing our Linux guest.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Discover how to write kernel code using the Loadable Kernel Module framework
  • Explore industry-grade techniques to perform efficient memory allocation and data synchronization within the kernel
  • Understand the essentials of key internals topics such as kernel architecture, memory management, CPU scheduling, and kernel synchronization

Description

Linux Kernel Programming is a comprehensive introduction for those new to Linux kernel and module development. This easy-to-follow guide will have you up and running with writing kernel code in next-to-no time. This book uses the latest 5.4 Long-Term Support (LTS) Linux kernel, which will be maintained from November 2019 through to December 2025. By working with the 5.4 LTS kernel throughout the book, you can be confident that your knowledge will continue to be valid for years to come. You’ll start the journey by learning how to build the kernel from the source. Next, you’ll write your first kernel module using the powerful Loadable Kernel Module (LKM) framework. The following chapters will cover key kernel internals topics including Linux kernel architecture, memory management, and CPU scheduling. During the course of this book, you’ll delve into the fairly complex topic of concurrency within the kernel, understand the issues it can cause, and learn how they can be addressed with various locking technologies (mutexes, spinlocks, atomic, and refcount operators). You’ll also benefit from more advanced material on cache effects, a primer on lock-free techniques within the kernel, deadlock avoidance (with lockdep), and kernel lock debugging techniques. By the end of this kernel book, you’ll have a detailed understanding of the fundamentals of writing Linux kernel module code for real-world projects and products.

Who is this book for?

This book is for Linux programmers beginning to find their way with Linux kernel development. If you’re a Linux kernel and driver developer looking to overcome frequent and common kernel development issues, or understand kernel intervals, you’ll find plenty of useful information. You’ll need a solid foundation of Linux CLI and C programming before you can jump in.

What you will learn

  • Write high-quality modular kernel code (LKM framework) for 5.x kernels
  • Configure and build a kernel from source
  • Explore the Linux kernel architecture
  • Get to grips with key internals regarding memory management within the kernel
  • Understand and work with various dynamic kernel memory alloc/dealloc APIs
  • Discover key internals aspects regarding CPU scheduling within the kernel
  • Gain an understanding of kernel concurrency issues
  • Find out how to work with key kernel synchronization primitives
Estimated delivery fee Deliver to Cyprus

Premium delivery 7 - 10 business days

€32.95
(Includes tracking information)

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Mar 19, 2021
Length: 754 pages
Edition : 1st
Language : English
ISBN-13 : 9781789953435
Vendor :
Linux Foundation
Languages :
Tools :

What do you get with Print?

Product feature icon Instant access to your digital eBook copy whilst your Print order is Shipped
Product feature icon Paperback book shipped to your preferred address
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
Estimated delivery fee Deliver to Cyprus

Premium delivery 7 - 10 business days

€32.95
(Includes tracking information)

Product Details

Publication date : Mar 19, 2021
Length: 754 pages
Edition : 1st
Language : English
ISBN-13 : 9781789953435
Vendor :
Linux Foundation
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
€18.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
€189.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts
€264.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just €5 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total 113.97
Mastering Embedded Linux Programming
€41.99
Linux Kernel Programming
€41.99
Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization
€29.99
Total 113.97 Stars icon

Table of Contents

18 Chapters
Section 1: The Basics Chevron down icon Chevron up icon
Kernel Workspace Setup Chevron down icon Chevron up icon
Building the 5.x Linux Kernel from Source - Part 1 Chevron down icon Chevron up icon
Building the 5.x Linux Kernel from Source - Part 2 Chevron down icon Chevron up icon
Writing Your First Kernel Module - LKMs Part 1 Chevron down icon Chevron up icon
Writing Your First Kernel Module - LKMs Part 2 Chevron down icon Chevron up icon
Writing Your First Kernel Module - LKMs Part 2
Technical requirements
A "better" Makefile template for your kernel modules
Configuring a "debug" kernel
Cross-compiling a kernel module
Setting up the system for cross-compilation
Attempt 1 – setting the "special" environment variables
Attempt 2 – pointing the Makefile to the correct kernel source tree for the target
Attempt 3 – cross-compiling our kernel module
Attempt 4  – cross-compiling our kernel module
Gathering minimal system information
Being a bit more security-aware
Licensing kernel modules
Emulating "library-like" features for kernel modules
Performing library emulation via multiple source files
Understanding function and variable scope in a kernel module
Understanding module stacking
Trying out module stacking
Passing parameters to a kernel module
Declaring and using module parameters
Getting/setting module parameters after insertion
Module parameter data types and validation
Validating kernel module parameters
Overriding the module parameter's name
Hardware-related kernel parameters
Floating point not allowed in the kernel
Auto-loading modules on system boot
Module auto-loading – additional details
Kernel modules and security – an overview
Proc filesystem tunables affecting the system log
The cryptographic signing of kernel modules
Disabling kernel modules altogether
Coding style guidelines for kernel developers
Contributing to the mainline kernel
Getting started with contributing to the kernel
Summary
Questions
Further reading
Section 2: Understanding and Working with the Kernel Chevron down icon Chevron up icon
Kernel Internals Essentials - Processes and Threads Chevron down icon Chevron up icon
Memory Management Internals - Essentials Chevron down icon Chevron up icon
Kernel Memory Allocation for Module Authors - Part 1 Chevron down icon Chevron up icon
Kernel Memory Allocation for Module Authors - Part 1
Technical requirements
Introducing kernel memory allocators
Understanding and using the kernel page allocator (or BSA)
The fundamental workings of the page allocator
Freelist organization
The workings of the page allocator
Working through a few scenarios
The simplest case
A more complex case
The downfall case
Page allocator internals – a few more details
Learning how to use the page allocator APIs
Dealing with the GFP flags
Freeing pages with the page allocator
Writing a kernel module to demo using the page allocator APIs
Deploying our lowlevel_mem_lkm kernel module
The page allocator and internal fragmentation
The exact page allocator APIs
The GFP flags – digging deeper
Never sleep in interrupt or atomic contexts
Understanding and using the kernel slab allocator
The object caching idea
Learning how to use the slab allocator APIs
Allocating slab memory
Freeing slab memory
Data structures – a few design tips
The actual slab caches in use for kmalloc
Writing a kernel module to use the basic slab APIs
Size limitations of the kmalloc API
Testing the limits – memory allocation with a single call
Checking via the /proc/buddyinfo pseudo-file
Slab allocator – a few additional details
Using the kernel's resource-managed memory allocation APIs
Additional slab helper APIs
Control groups and memory
Caveats when using the slab allocator
Background details and conclusions
Testing slab allocation with ksize() – case 1
Testing slab allocation with ksize() – case 2
Interpreting the output from case 2
Graphing it
Slab layer implementations within the kernel
Summary
Questions
Further reading
Kernel Memory Allocation for Module Authors - Part 2 Chevron down icon Chevron up icon
Kernel Memory Allocation for Module Authors - Part 2
Technical requirements
Creating a custom slab cache
Creating and using a custom slab cache within a kernel module
Creating a custom slab cache
Using the new slab cache's memory
Destroying the custom cache
Custom slab – a demo kernel module
Understanding slab shrinkers
The slab allocator – pros and cons – a summation
Debugging at the slab layer
Debugging through slab poisoning
Trying it out – triggering a UAF bug
SLUB debug options at boot and runtime
Understanding and using the kernel vmalloc() API
Learning to use the vmalloc family of APIs
A brief note on memory allocations and demand paging
Friends of vmalloc()
Specifying the memory protections
Testing it – a quick Proof of Concept
Why make memory read-only?
The kmalloc() and vmalloc() APIs – a quick comparison
Memory allocation in the kernel – which APIs to use when
Visualizing the kernel memory allocation API set
Selecting an appropriate API for kernel memory allocation
A word on DMA and CMA
Stayin' alive – the OOM killer
Reclaiming memory – a kernel housekeeping task and OOM
Deliberately invoking the OOM killer
Invoking the OOM killer via Magic SysRq
Invoking the OOM killer with a crazy allocator program
Understanding the rationale behind the OOM killer
Case 1 – vm.overcommit set to 2, overcommit turned off
Case 2 – vm.overcommit set to 0, overcommit on, the default
Demand paging and OOM
Understanding the OOM score
Summary
Questions
Further reading
The CPU Scheduler - Part 1 Chevron down icon Chevron up icon
The CPU Scheduler - Part 2 Chevron down icon Chevron up icon
Section 3: Delving Deeper Chevron down icon Chevron up icon
Kernel Synchronization - Part 1 Chevron down icon Chevron up icon
Kernel Synchronization - Part 2