End-to-End Example

By now, you’ve seen how the different parts of GitLab help you write, review, verify, secure, package, and deploy software. These features have been presented one at a time, so let’s put them all together in a single end-to-end example so you can see the whole process in one fell swoop. There won’t be any new material in this chapter, so it will be a great chance to review what you’ve learned in earlier chapters and see it all consolidated in one extended workflow.

This chapter is broken into many subsections, each showing how to use GitLab to help with a different part of the software development life cycle:

• Setting up your environment
• Writing code
• Establishing the pipeline infrastructure
• Verifying your code
• Securing your code
• Improving your pipeline
• Delivering your code to the right environment

Technical requirements

If you’d like to follow along as we develop a piece of software with GitLab, all you need is an account on a GitLab instance, whether Software-as-a-Service (SaaS) or self-hosted.

Setting up your environment

Before you start writing any code, you need to organize a few preliminary things. Specifically, you need to make a GitLab project for storing your code and running your pipelines, you need to make some GitLab issues to help plan and track your work, and you need to clone the project’s repository to your local computer so that you can write code using your favorite IDE instead of entirely within the GitLab GUI.

Making a GitLab project

Let’s rewind to the beginning of our journey with the Hats for Cats web app. We know we want to make a web app for our online feline accessories store, so let’s start by making a GitLab project to hold not only our web app’s code but also the additional GitLab components that we’ll be using as we build the app.

A note about the GUI

Throughout this chapter, we’ll continue our practice of showing you code snippets but generally not showing the steps for using the GitLab GUI...

Writing code

You’re almost at the point where you can start coding. First, you’ll need a Git branch to commit the code to. Then, you’ll need a MR so that you can see the results of pipeline tasks that run against that code, and also so that you can eventually merge the code into the main branch. Let’s go over those steps, and then make and push your first commit.

Creating a Git branch to work on

Now that we’re ready to add the login feature requested by our first assigned issue, we need a Git branch to commit to. We might as well name our branch after the title of the issue we’re working on:

git branch add-login-feature

Switch onto the new branch so any new commits end up on that branch and not on the main branch:

git checkout add-login-feature

At this point, we might as well push this branch up to the GitLab-hosted copy of the repository so that it exists in both places. For the first push, we need to use a slightly longer...

Establishing the pipeline infrastructure

You’ve stored some code in the repository, so now it’s time to set up a pipeline to run a variety of tasks to build, verify, and secure your code. In some cases, you might also want to set up a GitLab Runner to execute those pipeline tasks, although that task is usually taken care of for you by your GitLab administrator or the GitLab SaaS platform.

Creating a pipeline

Pushing commits to GitLab so that it can run CI/CD pipelines on our new code won’t work unless we define what tasks we’d like our pipeline to perform. We’ll add several tasks to our Hats for Cats project’s pipeline as we go through the rest of this chapter, but for now, let’s get a bare-bones pipeline in place.

Just as we did with our initial file, we could create the .gitlab-ci.yml pipeline configuration file locally, commit it, and push it up to GitLab, but since the GitLab GUI offers a handy editor dedicated to writing...

Verifying your code

Let’s configure your pipeline so that it can verify your code by running functional tests, Code Quality scanning, and fuzz tests.

Adding functional tests to the pipeline

Many teams start populating their pipelines by adding tasks to run automated functional tests to make sure their code is behaving the way it was designed to. You learned in previous chapters that there are many different sorts of functional tests. In this example, we’ll add some basic automated unit tests written with the pytest framework. Our project’s code is not yet complicated enough to require real unit tests, but for the sake of this example, we can add dummy tests so that GitLab can run them and display their results.

Before adding any tests, let’s make our login code slightly more complicated by adding a function that our tests can exercise, and a “to-do” comment that not only reminds us to flesh out this placeholder function later but also...

Securing your code

For this sample use case, you’re going to add four scanners to your pipeline: Static Application Security Testing (SAST), Secret Detection, Dependency Scanning, and License Compliance. You’ll also review how to add a third-party scanner.

Adding SAST to the pipeline

In general, adding a GitLab-provided security scanner to a pipeline is a trivial process. To enable SAST and make sure our Hats for Cats source code doesn’t contain security vulnerabilities, we simply need to include a new template in .gitlab-ci.yml on the add-login-feature branch. Add this line anywhere within the existing include: section, making sure that it’s indented correctly:

    - template: Security/SAST.gitlab-ci.yml

This enables SAST, but we also want to configure it so that it doesn’t scan our automated test file or our fuzz target file. The GitLab documentation tells us which variable to set to accomplish this. Add a new section...

Improving your pipeline

You’ve set up a pipeline to make sure your code is of high quality and doesn’t have security vulnerabilities. In many cases, you can stop there. However, for this sample use case, you’ll go a step further and look into using a DAG to speed up the pipeline. You’ll also see whether it’s worth splitting the pipeline’s configuration code into multiple files to improve readability and maintainability.

Using a DAG to speed up the pipeline

Our pipeline isn’t complicated enough to justify converting it into a DAG quite yet, but if we continue to add more jobs, we’ll eventually want to use DAGs for some or all of it for performance reasons. Let’s preview this by using the needs keyword now to add some DAG elements to our pipeline.

First, let’s say that we want the code_quality job to run only after the unit-tests job passes. After all, we might think that our code needs to work correctly before...

Delivering your code to the right environment

Your code is written, verified, and secured. The only step left is to deploy it.

Deploying the code

The final task of our pipeline is to deploy the Hats for Cats code to the right environment. Normally, we’d define separate jobs to deploy code to a review environment, the pre-production environment, or the production environment. Then, we would use GitLab’s rules: and if: keywords to control which one of those three jobs should run, depending on which Git branch the pipeline was running on.

To keep things simple in this example, let’s just walk through how to deploy it to the production environment. We’ll imagine that we want this to happen whenever we run the pipeline on the production branch.

As you learned in Chapter 8, there are countless techniques you can use to deploy code. Which technique you choose depends largely on what sort of environment you’re deploying to: an AWS EC2 VM, a Kubernetes...

Summary

In this chapter, we made a group and project to hold our code and other related components and made issues to plan and track our work. Then, we cloned the project’s repository to a local workstation so that we could write code using our favorite desktop tools. Then, we made a branch to commit our work to and an MR for that branch, linked it to an associated issue, and committed and pushed code for a new software feature. We set up a bare-bones CI/CD pipeline to which we can add a variety of tasks and registered specific runners for the project’s pipeline. We added automated unit tests to the pipeline to make sure the code satisfies its design specifications, as well as Code Quality scanning, and registered a special runner just for that scanner. We also added a fuzz test to the pipeline to find bugs in critical functions, and SAST to the pipeline to find security vulnerabilities in our code. We added Secret Detection to the pipeline to find any secrets that were...