Understanding GitLab’s CI/CD Pipeline Structure

By now, you have enough knowledge of Git and GitLab concepts to understand how developers can use those tools within the Create stage of the software development life cycle (SDLC) to create, review, and store code. You’ve also been introduced to some of the problems presented by the Verify, Package, and Release stages that come immediately after the Create stage. Now, it’s time to get to the meat (or tofu, if you prefer) of this book: how GitLab’s CI/CD pipelines can help solve those Verify, Package, and Release problems.

In this chapter, you’ll learn what continuous integration (CI) and continuous delivery (CD) mean, and how they fit into GitLab Flow. You’ll learn how to describe the different parts of a pipeline, including stages and jobs. You’ll see how those parts fit together, and how code flows through them. You’ll be shown how to view the overall status of a pipeline, and...

Technical requirements

As with the previous chapter, you’ll get the most out of this chapter if you’ve got an account on a GitLab instance (self-managed or Software-as-a-Service (SaaS)) that you can log in to and use to practice and experiment with the concepts discussed.

Defining the terms pipeline, CI, and CD

Since much of the power of GitLab comes from configuring CI/CD pipelines to do various things to and with your code, it’s critical to understand what a pipeline even is. So, an obvious place to start a discussion of this topic is to figure out exactly what we mean by pipeline, CI, and CD. We won’t start creating pipelines yet—that will come in a later chapter.

Understanding what a pipeline is

A GitLab CI/CD pipeline is a series of steps that are performed on your files whenever you commit edits to the GitLab-hosted copy of a repository. A lot is going on in that sentence, so let’s take a more careful look at each part of it.

What do we mean by “a series of steps”? You can think of these steps as tasks that are performed on your files. For example, you may want to run various tests or scanners on your files to make sure your code is well-written, is free from security vulnerabilities, uses appropriately...

Parts of a pipeline – stages, jobs, and commands

That’s the big picture of what a GitLab CI/CD pipeline is – how the CI portion of a pipeline differs from the CD portion of the same pipeline and why pipelines are such an important part of the SDLC. Let’s zoom in a little and take a look at the structure of a pipeline in more detail. In particular, how is a pipeline put together from stages and jobs?

Stages

Every pipeline consists of one or more stages. A stage is a collection of pipeline tasks that are thematically related. For example, these are probably the three most commonly used stages:

• Build: This stage holds tasks that compile and/or package your source code into a deployable format.
• Test: This stage holds tasks that run automated tests, code quality scans and linters, and possibly security scans.
• Deploy: This stage sends your code to the appropriate environment, depending on what Git branch or Git tag the pipeline is running...

Running GitLab CI/CD pipelines

Whenever a project’s pipeline runs, it’s running on some version of that project’s files. This means that in the CI portion of the pipeline, it runs automated tests and scans on just one version of your files. Then, in the CD portion, it deploys that same version of the files to the appropriate environment. You will also see this described as a pipeline running “against” a version of your project’s files.

The point of pipelines is to check the status of your code – and then deploy that code – every time you make changes to it. So, running a project’s pipeline on yesterday’s version of your code may produce one set of results, while running the pipeline against today’s version of the code may generate very different results, even though the pipeline consists of the same stages, jobs, and commands. Between yesterday and today, you may have added new automated tests, introduced...

Reading GitLab CI/CD pipeline statuses

Not only does each pipeline instance have a pass/fail status, but each stage within the pipeline instance has a pass/fail status, and each job within any stage has a pass/fail status as well. There are more statuses available than just passed or failed. Here are some of the most commonly seen values:

• running: The pipeline, stage, or job is in progress.
• pending: Waiting for resources to become available to start a job.
• skipped: When an earlier stage fails, all later stages are skipped by default.
• canceled: Users can cancel any job or pipeline while it’s running.

In Figure 4.3, you saw how the list of pipeline instances shows the status not only of each pipeline instance but also of the stages within each pipeline. In Figure 4.4, you saw how you can zoom in on an individual pipeline instance to see the status of all the jobs within each of the pipeline’s stages. GitLab lets you zoom in even further to see...

Configuring GitLab CI/CD pipelines

We’ve mentioned that you can configure your project’s CI/CD pipeline to define its stages, jobs, and commands. But how do you do that? All CI/CD pipeline configuration happens within a file called .gitlab-ci.yml, which lives in the root of your project’s repository. Look through any public GitLab project, and you’re sure to see a file with that name that determines what happens in that project’s pipeline.

Every .gitlab-ci.yml file uses a domain-specific language that consists of keywords, values, and some syntactical glue. Some keywords define stages and jobs within those stages. Other keywords configure jobs to do different things within the pipeline. Still, other keywords set variables, specify Docker images for jobs, or affect the overall pipeline in various ways. This domain-specific language is rich enough to let you do just about anything you’d like in your CI/CD pipelines, but not so rich as to be...

Summary

Now that you have a good grasp of the purpose and structure of GitLab CI/CD pipelines, let’s review the concepts we covered in this chapter.

Pipelines are a series of steps that are performed against code in your project’s Git repository. Each project has just one pipeline, although the various steps that make up a project’s pipeline can be run or suppressed, depending on which Git branch or Git tag the pipeline is running against. The term “pipeline” is sometimes used to mean the overall set of tasks that will be run on a project’s code, while other times, it’s used to mean a single instance or run of that pipeline against a particular version of the repository’s files.

The CI, or continuous integration, portion of a pipeline answers the question, is the code good? It typically consists of some combination of automated tests, security scans, license compliance checks, and code quality checks. The CI steps of a pipeline...