Linux Service Management Made Easy with systemd

By Donald A. Tevault

Early Access

This is an Early Access product. Early Access chapters haven’t received a final polish from our editors yet. Every effort has been made in the preparation of these chapters to ensure the accuracy of the information presented. However, the content in this book will evolve and be updated during the development process.

Learn more
    Advance your knowledge in tech with a Packt subscription

  • Instant online access to over 7,500+ books and videos
  • Constantly updated with 100+ new titles each month
  • Breadth and depth in over 1,000+ technologies
  1. 1 Understanding the Need for systemd

About this book

systemd is a new type of Linux init system for today’s high-performance, multi-CPU, and multi-core hardware that is now used on all major enterprise-grade Linux distros. The main goal of this Linux systemd book is to help you get an in-depth understanding of systemd to set up your servers securely and efficiently.

This book starts by explaining systemd management, which will help you manage your servers effectively. You’ll then learn how to edit and create your own systemd units, which will be particularly helpful if you need to create custom services or timers and add features or security to an existing service. Next, you'll understand how to analyze and fix boot-up challenges and set system parameters. Later, you'll come across cgroups, that'll help you control system resource usage for both processes and users. The book also shows you how cgroups are structured, the differences between cgroups Version 1 and 2, and how to set resource limits on both. Finally, you'll learn about the systemd way of performing time-keeping, networking, logging, and login management. You'll discover how to configure servers accurately and gather system information to analyze system security and performance.

By the end of this Linux book, you’ll be able to efficiently manage all aspects of a server running the systemd init system.

Publication date:
February 2022
Publisher
Packt
Pages
373
ISBN
9781801811644

 

1 Understanding the Need for systemd

In this first chapter, we’ll first briefly look at the history of Linux init systems. We’ll then look at the shortcomings of the legacy init systems and why certain Linux engineers felt the need to develop a new type of init system. Finally, we’ll look at the controversy that has surrounded systemd. For easy reference, here’s a list of the topics:

  • The history of Linux init systems
  • The shortcomings of SysV init and upstart
  • The advantages of systemd
  • The systemd controversy

So, with the introductory comments out of the way, let’s jump in.

 

Technical requirements

For this chapter, all you need is a Linux virtual machine that runs systemd. As you read through this chapter, you might want to look at some of the files on the virtual machine.

 

The history of Linux init systems

So, what is an init system? Well, init is short for initialization. An init system, then, initializes the operating system upon bootup. After the bootup has completed, the init system will continue working, managing system processes and services. Each system process is assigned a process ID number, or PID. The init process is always PID 1, and every other process that gets started on the system is either a child or a grandchild of the init process.

For many years, the SysV Init system was the primary init system for Linux-based operating systems (SysV is short for System 5. The V is the Roman numeral for 5). SysV init was originally developed by Bell Labs engineers for the Unix operating system, all the way back in the early 1970s. (At that time, I was a young pup in junior high school, and I still had a full head of hair.)

Note

There are actually a few more Linux init systems besides the ones that I’m mentioning here. But these were the most commonly used ones in the pre-systemd days.

SysV init worked well in its day, but it was never perfect. Nowadays, with new high-performance hardware, SysV init has shown both its age and its deficiencies. The first attempt to come up with something better occurred in July 2009, when Ubuntu engineers released the first version of the upstart init system. Although it was better than SysV, it still had its share of problems, especially the early versions which were quite buggy.

 

The shortcomings of SysV Init and upstart

The first problem with SysV is that of its rather lengthy boot-up times. When you boot up a SysV machine, all of its services have to start up in sequential order. That might not be so bad on a normal desktop machine, but it can be a bit problematic on a server that needs to run lots of services. In that case, each service would have to wait its turn to start, which could take a while.

The next problem with SysV is its complexity. Instead of simple, easy-to-understand configuration files, SysV does everything with complex Bash shell scripts. The init scripts that control system services all have to be assigned a priority number, so that services will start and stop in the proper order. Take, for example, the init script that starts the Apache webserver on a CentOS 5 machine. First, we can see that it’s a fairly lengthy script, as shown here:

[[email protected] init.d]$ pwd
/etc/init.d
[[email protected] init.d]$ ls -l httpd
-rwxr-xr-x 1 root root 3523 Sep 16  2014 httpd
[[email protected] init.d]$ wc -l httpd
131 httpd
[[email protected] init.d]$

You can see from the wc -l output that it consists of 131 lines. As you can see here, 37 of those lines are comments, which still leaves us with 94 lines of actual code:

[[email protected] init.d]$ grep ^# httpd | wc -l
37
[[email protected] init.d]$

Look inside, and you’ll see that it’s quite complex and convoluted. Here’s just the first part of it:

Figure 1.1 – And old-fashioned SysV Init script

Toward the end of the script, you’ll see the code that stops, starts, restarts, and reloads the Apache daemon, as shown here:

Figure 1.2 – The start, stop, restart, reload section of an init script

This code, or code similar to this, has to be in every init script so that the human user can control the daemon. To complicate things even more, developers didn’t always write this code consistently for different programs. So, for example, a status display for one daemon didn’t always look the same as the status display for another daemon.

Then, there’s the problem of inconsistent implementation across the different families of Linux distributions. With SysV, there were at least three different methods of implementation. Red Hat-type distributions used one method, Debian-type distributions used another method, and Slackware-type distributions use yet another. For example, the Red Hat way of controlling services required using the service and chkconfig commands. When working with Debian-type systems, I always used to have to look up the service management commands, because I could never remember them. With Slackware, you don’t have any service management commands. To enable or disable a service on a Slackware machine, you just set or remove the executable permission from the appropriate init script.

Runlevels were also a source of confusion, because each family of distros had its own set of runlevel definitions. For example, for the graphical runlevel, the following is the case:

  • The Red Hat family used runlevel 5.
  • The Slackware family uses runlevel 4.
  • The Debian family used no specific runlevel for either text mode or graphical mode. Instead, you enabled or disabled graphical mode by enabling or disabling the XServer daemon.

So, you can see that this was all quite confusing, especially for anyone who worked in a mixed environment. It should be fairly obvious that we needed something that was a bit less confusing.

As if this weren’t enough, there was also the issue of performance. SysV worked well in its day, when computing hardware was more primitive. But, on modern hardware with multiple CPUs that each have multiple cores, we need something a bit more robust. Ubuntu’s Upstart was supposed to fix this, but it didn’t quite live up to its promise. Nowadays, Upstart is completely dead, but there are still some diehards who refuse to give up SysV. In the enterprise, systemd is king.

 

The advantages of systemd

We’ve just seen the problems with SysV and Upstart. Now, let’s look at what makes systemd better.

systemd simplicity

In contrast to SysV, systemd is really quite simple to configure. For example, look at how short the Apache service file is on a CentOS 7 machine with systemd:

[[email protected] ~]$ cd /lib/systemd/system
[[email protected] system]$ ls -l httpd.service
-rw-r--r--. 1 root root 752 Jun 26  2018 httpd.service
[[email protected] system]$ wc -l httpd.service
22 httpd.service
[[email protected] system]$

There are only 22 lines, and 5 of those lines are comments, as you can see here:

Figure 1.3 – A systemd service file

I’ll explain everything in the systemd files later. For now, I just want to show you that a systemd service file is much simpler than a SysV Init script. (As we’ll soon see in the upcoming chapters, it’s easier to learn how to use the systemd directives than it is to learn how to write shell-scripting code for init scripts.)

systemd consistency

The next systemd advantage is its consistency. Yes, boys and girls, you no longer have to remember multiple sets of system management commands for multiple families of Linux distributions. Instead, you’ll now use the same commands on all Linux distributions that use systemd. So, this eliminates a major source of frustration for administrators, and for anyone who’s studying to take a Linux certification exam.

systemd performance

In contrast to SysV, systemd can start services in parallel, rather than just one at a time in sequence. This makes for much quicker boot-up times than for SysV. Once the machine is booted, performance is more robust than that of SysV.

With systemd, we have a much cleaner way of killing processes. For example, if you needed to use the kill command to forcefully terminate the Apache webserver service on a SysV machine, you would only terminate the Apache process itself. If the webserver process had spawned any child processes due to running CGI scripts, for example, those processes would continue on for a while longer as zombie processes. But, when you kill a service with systemd, all processes that are associated with that service will also get terminated.

systemd security

An added bonus is that you can configure systemd service files to control certain aspects of system security. Here are some of the things that you can do:

  • You can create a systemd service that can restrict access to or from certain directories, or that can only access or be accessed from certain network addresses.
  • By using namespaces, you can effectively isolate services from the rest of the system. This also allows you to create containers without having to run Docker.
  • You can use cgroups to limit resource usage. This can help prevent certain types of denial-of-service attacks.
  • You can specify which root-level kernel capabilities that a service is allowed to have.

With all this, you can make systemd somewhat emulate a mandatory access control system, such as SELinux or AppArmor.

All the way around, systemd is much better than any init system that came before it. But it hasn’t made everyone happy.

 

The systemd controversy

If you’ve been in the computer world for any length of time, you may have seen that we geeks can get quite passionate about our operating systems. In the early 1990s, I finally replaced my text mode-only 8088 machine with one that could run a graphical interface. I first gave Windows 3.1 a try, and quickly decided that I really hated it. So, I bought a copy of OS/2, which I liked much better and ran for quite a few years on my home-built 486 machine. But, all of my geek buddies at work were big Windows fans, and they kept arguing with me about how much better Windows is. I thought that they were all crazy, and we kept getting into some rather heated arguments.

Then, when I got into Linux, I quickly learned that you don’t want to go into any Linux forum and ask which Linux distribution is the best for a newbie to start with. All that does is start fights, leaving the poor newbie more confused than ever. And now, the fight is over whether or not systemd is a good thing. Here are some of the objections:

  • By trying to do too much, systemd violates the Unix concept of having each utility just do one thing but having it do it well.
  • It’s controlled by a large corporation (Red Hat).
  • It’s a security problem.
  • Its journald component saves system logs to a binary format, which some people believe is more easily corrupted than the plain-text files that rsyslog creates.

If you look at things objectively, you might see that the objections aren’t so bad:

  • Yes, the systemd ecosystem includes more than just the init system. It also includes network, bootloader, logging, and log-in components. But those components are all optional, and not all Linux distributions use them in a default setup.
  • It was created primarily by Red Hat, and the project leader is a Red Hat employee. But Red Hat released it under a free-as-in-speech software license, which means that no one company can ever take full control of it. Even if Red Hat were to suddenly decide that future versions of systemd were to be proprietary, the free code is still out there, and someone would fork it into a new free version.
  • Yes, there have been some security bugs in systemd. But that’s also true of OpenSSL, the Bash shell, and even the Linux kernel itself. To complain about systemd security would only be valid if the bugs hadn’t gotten fixed.
  • The journald component does create log files in a binary format. But it’s still possible to run rsyslog on systemd distributions, and most do. Some distributions, such as the Red Hat Enterprise Linux 8 family, use journald to gather system information and then just have journald pass the information to rsyslog in order to create normal text files. So, with RHEL 8, we have the best of both worlds.

Soon after the release of systemd, some people who had never even tried it put up blog posts that explained why systemd was pure evil and that they would never use it. A few years ago, I created a systemd tutorial playlist on my BeginLinux Guru channel on YouTube. The first video is called Why systemd? Quite a few people left comments about why they would never use systemd and said that they would change to either a non-systemd Linux distribution or to a FreeBSD-type distribution in order to avoid it.

The bottom line is this: all enterprise-grade Linux distributions now use systemd. So, I think that it might be here to stay.

 

Summary

In this first chapter, we’ve looked at the history of the most common Linux init systems. We’ve seen the ways in which the legacy init systems are deficient, and we’ve seen why systemd is a much better replacement. We wrapped things up by looking at the objections against systemd.

One of the challenges of learning systemd is that, until now, there hasn’t been any real comprehensive documentation about it. There’s basic usage documentation on the Red Hat website, but it doesn’t even cover all components of the systemd ecosystem. There are only two systemd-specific books that I could find, which are a few years old. (One book is specific for Fedora, the other is specific for Ubuntu.) Even those books leave some things out. So, the challenge I’ve set for myself is to create a comprehensive, hands-on guide for all things systemd. In the chapters that follow, I’ll do my best to accomplish that goal.

In the next chapter, we’ll go on a quick tour of the systemd directories and files. I’ll see you there.

 

Questions

  1. Who created the original SysV init system?A. Bell LabsB. Red HatC. DebianD. Ubuntu
  2. Which of the following is true about SysV?A. It’s a modern, robust init system.B. When booting a machine, it can start services in parallel.C. When booting a machine, it can only start services sequentially.D. It has security features that systemd doesn’t have.
  3. Which of the following is not true about systemd?A. It has security features that can somewhat emulate a mandatory access control system.B. It can start services in parallel.C. It can use cgroups to limit resource usage.D. It’s a legacy system that needs to be replaced.
 

Answers

  1. A
  2. C
  3. D
 

About the Author

  • Donald A. Tevault

    Donald A. Tevault - but you can call him Donnie - got involved with Linux way back in 2006, and has been working with it ever since. He holds the Linux Professional Institute Level 3-Security certification, and the GIAC Incident Handler certification. Donnie is a professional Linux trainer, and thanks to the magic of the internet, teaches Linux classes literally the world over from the comfort of his living room. He's also a Linux security researcher for an IoT security company.

    Browse publications by this author
Linux Service Management Made Easy with systemd
Unlock this book and the full library for $5 a month*
Start now