2

Clang Architecture

In this chapter, we will examine the internal architecture of Clang and its relationship with other LLVM components. We will begin with an overview of the overall compiler architecture, with a specific focus on the clang driver. As the backbone of the compiler, the driver runs all compilation phases and controls their execution. Finally, we will concentrate on the frontend portion of the Clang compiler, which includes lexical and semantic analysis, and produces an Abstract Syntax Tree (AST ) as its primary output. The AST forms the foundation for most Clang tools, and we will examine it more closely in the next chapters.

The following topics will be covered in this chapter:

• Compiler overview

• Clang driver overview, including an explanation of the compilation phases and their execution

• Clang frontend overview covering the preprocessing step, parsing, and semantic analysis

2.1 Technical requirements

The source code for this chapter is located in the chapter2 folder of the book’s GitHub repository: https://github.com/PacktPublishing/Clang-Compiler-Frontend-Packt/tree/main/chapter2.

2.2 Getting started with compilers

Despite the fact that compilers are used to translate programs from one form to another, they can also be considered large software systems that use various algorithms and data structures. The knowledge obtained by studying compilers can be used to design other scalable software systems as well. On the other hand, compilers are also a subject of active scientific research, and there are many unexplored areas and topics to investigate.

You can find some basic information about the internal structure of a compiler here. We will keep it as basic as possible so the information applies to any compiler, not just Clang. We will briefly cover all phases of compilation, which will help to understand Clang’s position in the overall compiler architecture.

2.2.1 Exploring the compiler workflow

The primary function of a compiler is to convert a program written in a specific programming language (such as C/C++ or FORTRAN) into a format that can be executed...

2.3 Clang driver overview

When discussing compilers, we typically refer to a command-line utility that initiates and manages the compilation process. For example, to use the GNU Compiler Collection, one must call gcc to start the compilation process. Similarly, to compile a C++ program using Clang, one must call clang as the compiler. The program that controls the compilation process is known as the driver. The driver coordinates different stages of compilation and connects them together. In the book, we will be focusing on LLVM and using Clang as the driver for the compilation process.

It may be confusing for readers that the same word, ”Clang,” is used to refer to both the compiler frontend and the compilation driver. In contrast, with other compilers, where the driver and C++ compiler can be separate executables, ”Clang” is a single executable that functions as both the driver and the compiler frontend. To use Clang as the compiler frontend only, the special...

2.4 Clang frontend overview

It’s evident that the Clang compiler toolchain conforms to the pattern widely described in various compiler books [1, 18]. However, Clang’s frontend part diverges significantly from a typical compiler frontend. The primary reason for this distinction is the complexity of the C++ language. Some features, such as macros, can modify the source code itself, while others, such as typedef, can influence the kind of token. Clang can also generate output in a variety of formats. For instance, the following command generates an aesthetically pleasing HTML view of the program shown in Figure 2.5:

$ <...>/llvm-project/install/bin/clang -cc1 -emit-html max.cpp

Take note that we pass the argument to emit the HTML form of the source program to the Clang frontend, specified with the -cc1 option. Alternatively, you can pass an option to the frontend via the -Xclang option, which requires an additional argument representing...

2.5 Summary

In this chapter, we have acquired a basic understanding of compiler architecture and delved into the various stages of the compilation process, with a focus on the Clang driver. We have explored the internals of the Clang frontend, studying the Preprocessor that transforms a program into a set of tokens, and the Parser, which interacts with a component called ’Sema’. Together, these elements perform syntax and semantic analysis.

The upcoming chapter will center on the Clang Abstract Syntax Tree (AST)—the primary data structure employed in various Clang tools. We will discuss its construction and the methods for traversing it.

2.6 Further reading

• Working Draft, Standard for Programming Language C++: https://eel.is/c++draft/

• “Clang” CFE Internals Manual: https://clang.llvm.org/docs/InternalsManual.html

• Keith Cooper and Linda Torczon: Engineering A Compiler, 2012 [18]