This chapter is an introduction to system programming, exploring a range of topics from its original definition to how it has shifted in time with system evolution. This chapter provides some basic concepts and an overview of Unix and its resources, including the kernel and the application programming interfaces (API). Many of these concepts are defined here and are used in the rest of the book.
The following topics will be covered in this chapter:
- What is system programming?
- Application programming interfaces
- Understanding how the protection ring works
- An overview of system calls
- The POSIX standard
This chapter does not require you to install any special software if you're on Linux.
If you are a Windows user, you can install the Windows Subsystem for Linux (WSL). Follow these steps in order to install WSL:
- Open PowerShell as administrator and run the following:
Enable-WindowsOptionalFeature -Online -FeatureName Microsoft-Windows-Subsystem-Linux
- Restart your computer when prompted.
- Install your favorite Linux distribution from the Microsoft Store.
Over the years, the IT landscape has shifted dramatically. Multicore CPUs that challenge the Von Neumann machine, the internet, and distributed systems are just some of the changes that occurred in the last 30 years. So, where does system programming stand in this landscape?
Let's start with the standard textbook definition first.
System programming (or systems programming) is the activity of programming computer system software. The primary distinguishing characteristic of system programming when compared to application programming is that application programming aims to produce software that provides services directly to the user (for example, a word processor), whereas system programming aims to produce software and software platforms that provide services to other software, and are designed to work in performance-constrained environments, for example, operating systems, computational science applications, game engines and AAA video games, industrial automation, and software as a service applications.
The definition highlights two main concepts of what system applications are as follows:
- Software that is used by other software, not directly by the final user.
- The software is hardware aware (it knows how the hardware works), and is oriented toward performance.
This makes it possible to easily recognize as system software operating system kernels, hardware drivers, compilers, and debuggers, and not as system software, a chat client, or a word processor.
Historically, system programs were created using Assembly and C. Then came the shells and the scripting languages that were used to tie together the functionality offered by system programs. Another characteristic of system languages was the control of the memory allocation.
Other languages, such as Go, proved that garbage collection and performance are not mutually exclusive. In particular, Go managed to replace its own memory allocator written in C with a native version written in Go in release 1.5, improving it to the point where the performance was comparable.
In the meantime, systems started to get distributed and the applications started to get shipped in containers, orchestrated by other system software, such as Kubernetes. These systems are meant to sustain huge throughput and achieve it in two main ways:
- By scaling—augmenting the number or the resources of the machines that are hosting the system
- By optimizing the software in order to be more resource effective
Some of the practices of system programming—such as having an application that is tied to the hardware, performance oriented, and working on an environment that is resource constrained—are an approach that can also be valid when building distributed systems, where constraining resource usage allows the reduction of the number of instances needed. It looks like system programming is a good way of addressing generic software engineering problems.
This means that learning the concept of system programming with regards to using the resource of the machine efficiently—from memory usage to filesystem access—will be useful when building any type of application.
APIs are series subroutine definitions, communication protocols, and tools for building software. The most important aspects of an API are the functionalities that it offers, combined with its documentation, which facilitates the user in the usage and implementation of the software itself in another software. An API can be the interface that allows an application software to use a system software.
An API usually has a specific release policy that is meant to be used by a specific group of recipients. This can be the following:
- Private and for internal use only
- Partner and usable by determined groups only—this may include companies that want to integrate the service with theirs
- Public and available for every user
We'll see that there are several types of APIs, from the ones used to make different application software work together, to the inner ones exposed by the operating system to other software.
The protection ring, also referred to as hierarchical protection domains, is the mechanism used to protect a system against failure. Its name is derived from the hierarchical structure of its levels of permission, represented by concentric rings, with privilege decreasing when moving to the outside rings. Between each ring there are special gates that allow the outer ring to access the inner ring resources in a restricted manner.
The number and order of rings depend on the CPU architecture. They are usually numbered with decreasing privilege, making ring 0 the most privileged one. This is true for i386 and x64 architecture that use four rings (from ring 0 to ring 3) but it's not true for ARM, which uses reverse order (from EL3 to EL0). Most operating systems are not using all four levels; they end up using a two level hierarchy—user/application (ring 3) and kernel (ring 0).
A software that runs under an operating system will be executed at user (ring 3) level. In order to access the machine resources, it will have to interact with the operating system kernel (that runs at ring 0). Here's a list of some of the operations a ring 3 application cannot do:
- Modify the current segment descriptor, which determines the current ring
- Modify the page tables, preventing one process from seeing the memory of other processes
- Use the LGDT and LIDT instructions, preventing them from registering interrupt handlers
- Use I/O instructions such as in and out that would ignore file permissions and read directly from disk
The access to the content of the disk, for instance, will be mediated by the kernel that will verify that the application has permission to access the data. This kind of negotiation improves security and avoids failures, but comes with an important overhead that impacts the application performance.
Some applications can be designed to run directly on the hardware without the framework provided by an operating system. This is true for real-time systems, where there is no compromise on response times and performance.
System calls are the way operating systems provide access to the resources for the applications. It is an API implemented by the kernel for accessing the hardware safely.
There are some categories that we can use to split the numerous functions offered by the operating system. These include the control of the running applications and their flow, the filesystem access, and the network.
This type of services includes load, which adds a program to memory and prepares for its execution before passing control to the program itself, or execute, which runs an executable file in the context of a pre-existing process. Other operations that belong to this category are as follows:
- end and abort—the first requires the application to exit while the second forces it.
- CreateProcess, also known as fork on Unix systems or NtCreateProcess in Windows.
- Terminate process.
- Get/set process attributes.
- Wait for time, wait event, or signal event.
- Allocate and free memory.
The handling of files and filesystems belongs to file management system calls. There are create and delete files that make it possible to add or remove an entry from the filesystem, and open and close operations that make it possible to gain control of a file in order to execute read and write operations. It is also possible to read and change file attributes.
Device management handles all other devices but the filesystem, such as frame buffers or display. It includes all operations from the request of a device, including the communication to and from it (read, write, seek), and its release. It also includes all the operations of changing device attributes and logically attaching and detaching them.
Reading and writing the system date and time belongs to the information maintenance category. This category also takes care of other system data, such as the environment. Another important set of operations that belongs here is the request and the manipulation of processes, files, and device attributes.
All the network operations from handling sockets to accepting connections fall into the communication category. This includes the creation, deletion, and naming of connections, and sending and receiving messages.
Windows has a series of different system calls that cover all the kernel operations. Many of these correspond exactly with the Unix equivalent. Here's a list of some of the overlapping system calls:
In order to ensure consistency between operating systems, IEEE formalized some standards for operating systems. These are described in the following sections.
Portable Operating System Interface (POSIX) for Unix represents a series of standards for operating system interfaces. The first version dates back to 1988 and covers a series of topics like filenames, shells, and regular expressions.
There are many features defined by POSIX, and they are organized in four different standards, each one focusing on a different aspect of the Unix compliance. They are all named POSIX followed by a number.
POSIX.1 is the 1988 original standard, which was initially named POSIX but was renamed to make it possible to add more standards to the family without giving up the name. It defines the following features:
- Process creation and control
- Floating point exceptions
- Segmentation/memory violations
- Illegal instructions
- Bus errors
- File and directory operations
- C library (standard C)
- I/O port interface and control
- Process triggers
POSIX.1b focuses on real-time applications and on applications that need high performance. It focus on these aspects:
- Priority scheduling
- Real-time signals
- Clocks and timers
- Message passing
- Shared memory
- Asynchronous and synchronous I/O
- Memory locking interface
POSIX.1c introduces the multithread paradigm and defines the following:
- Thread creation, control, and cleanup
- Thread scheduling
- Thread synchronization
- Signal handling
POSIX.2 specifies standards for both a command-line interpreter and utility programs as cd, echo, or ls.
Not all operating systems are POSIX compliant. Windows was born after the standard and it is not compliant, for instance. From a certification point of view, macOS is more compliant than Linux, because the latter uses another standard built on top of POSIX.
Most Linux distributions follow the Linux Standard Base (LSB), which is another standard that includes POSIX and much more, focusing on maintaining the inter-compatibility between different Linux distributions. It is not considered officially compliant because the developers didn't go into the process of certification.
However, macOS became fully compatible in 2007 with the Snow Leopard distribution, and it has been POSIX-certified since then.
Windows is not POSIX compliant, but there are many attempts to make it so. There are open source initiatives such as Cygwin and MinGW that provide a less POSIX-compliant development environment, and support C applications using the Microsoft Visual C runtime library. Microsoft itself has made some attempts at POSIX compatibility, such as the Microsoft POSIX subsystem. The latest compatibility layer made by Microsoft is the Windows Linux Subsystem, which is an optional feature that can be activated in Windows 10 and has been well received by developers (including myself).
In this chapter, we saw what system programming means—writing system software that has some strict requirements, such as being tied to the hardware, using a low-level language, and working in a resource-constrained environment. Its practices can be really useful when building distributed systems that normally require optimizing resource usage. We discussed APIs, definitions that allows software to be used by other software, and listed the different types—the ones in the operating system, libraries and frameworks, and remote and web APIs.
We analyzed how, in operating systems, the access to resources is arranged in hierarchical levels called protection rings that prevent uncontrolled usage in order to improve security and avoid failures from the applications. The Linux model simplifies this hierarchy to just two levels called user and kernel space. All the applications are running in the user space, and in order to access the machine's resources they need the kernel to intercede.
Then we saw one specific type of API called system calls that allows the applications to request resources to the kernel, and mediates process control, access and management of files, and devices and network communications.
We gave an overview of the POSIX standard, which defines Unix system interoperability. Among the features defined, there are also the C API, CLI utilities, shell language, environment variables, program exit status, regular expressions, directory structures, filenames, and command-line utility API conventions.
In the next chapter, we will explore the Unix operating system resources such as the filesystem and the Unix permission model. We will look at what processes are, how they communicate with each other, and how they handle errors.
- What is the difference between application and system programming?
- What is an API? Why are APIs so important?
- Could you explain how protection rings work?
- Can you make some examples of what cannot be done in user space?
- What's a system call?
- Which calls are used in Unix to manage a process?
- Why is POSIX useful?
- Is Windows POSIX compliant?