Verifying Your Code

For most projects, the first thing a GitLab CI/CD pipeline should do is verify the code. Different projects will rely on different tasks to perform this critical step, but they usually involve some combination of checking the code quality and running automated functional tests. As a prerequisite for certain kinds of verification, some projects will need to build their code first. This chapter focuses on building and then verifying your code.

We’ll first discuss whether building the code is necessary, and if so, how to configure a GitLab CI/CD pipeline to carry out that task. Then, we’ll talk about how to use a pipeline to run GitLab’s built-in code quality scanner. Next, we’ll explain how to run automated functional tests within a pipeline. Then, we’ll cover a fascinating variety of automated testing called fuzz testing, which can find problems that traditional automated functional tests might miss. We’ll touch on GitLab...

Technical requirements

As with the previous chapters, you’ll get the most out of this chapter if you’ve got an account on a GitLab instance (self-managed or Software-as-a-Service) that you can log in to and use for practicing and experimenting with the concepts discussed.

Building code in a CI/CD pipeline

At the risk of oversimplifying some of the mechanics that happen behind the scenes when you run software, we can generally think of interpreted computer languages such as Python or Ruby as executing raw source code, whereas compiled languages such as Java, C, or C# must convert that source code into a runnable form by compiling it, and then execute the compiled version of the program.

This is an important distinction to keep in mind when configuring a pipeline to verify your code because it means that if your project contains any code written in a compiled language (even if it’s only a small portion of your overall project), you probably need to include a build job in your pipeline before any verification jobs take place. We say probably because some of the jobs that typically run during the verification stage of a pipeline (for example, Code Quality) look directly at source code, whereas others interact with code as it runs. So, if your...

Checking code quality in a CI/CD pipeline

One of the many scanners that GitLab makes available to CI/CD pipelines is a special feature that makes sure your project’s code adheres to certain quality standards. GitLab calls this feature, unsurprisingly, Code Quality. If you’ve used any sort of linting tool before, you can think of this feature as a turbocharged linter.

The Code Quality feature relies on an outside service called Code Climate. Although this service can scan code written in all the major computer languages, it can’t handle every language out there. You can refer to Code Climate’s official documentation to see a list of supported languages, but rest assured that it works just fine with Java, Python, Ruby, JavaScript, and most other commonly used languages.

What sorts of problems does the Code Quality feature look for? The general categories it’s interested in include performance, style, complexity, security, and smells (i.e., patterns...

Running automated functional tests in a CI/CD pipeline

One of the most common tasks in a CI/CD pipeline is running automated functional tests to make sure your code does what it’s supposed to do. For example, you might want to use the pytest framework to run a collection of unit tests written in Python to test your Python-based Hats for Cats app. Let’s see how to do that with GitLab.

Note

If you’re not familiar with pytest, don’t worry. The syntax for pytest unit tests is extremely simple and can be understood by anyone with even a little experience of writing automated tests in any language.

Enabling automated functional tests

Imagine that you’ve written three pytest-based unit tests to make sure the Hats for Cats app’s login feature works as expected. You might have a file called test/test_login.py with these contents:

def test_login():
    # add code that tries to log in with good credentials
  ...

Fuzz testing in a CI/CD pipeline

Fuzz testing is an alternative, less traditional way of finding bugs in your code. Put succinctly, this advanced testing technique sends semi-random data to your code’s functions in an effort to trigger bugs. Although it takes a little more work to set up than the other scanners, it can pay off by spotting bugs that you probably never would have found using other methods.

Reminder about GitLab versions and features

Fuzz testing, like many other features discussed throughout the book, is only available if you’re using GitLab with an Ultimate license. You can find out whether your license tier includes a particular feature by looking up that feature in the official GitLab documentation. Features are often made available in lower tiers after they’ve been restricted to higher tiers for a few years.

There are two ways of performing fuzz testing in GitLab: coverage-guided fuzz testing and web API fuzz testing. In this book, we...

Checking accessibility in a CI/CD pipeline

Not all applications include web interfaces, but whenever you do write a web app, we strongly recommend you use your GitLab CI/CD pipeline to make sure your interface is accessible for people with a range of disabilities. Fortunately, GitLab makes it easy to test your website against the Web Content Accessibility Guidelines (WCAG) laid out by the World Wide Web Consortium.

These guidelines address a wide assortment of characteristics of websites that could cause accessibility problems. Here are just a few of the things that the WCAG covers:

• Pages that require scrolling both vertically and horizontally
• HTML heading tags such as <H1> that contain no text
• Text that doesn’t contrast strongly enough with its background
• Images that lack an alternative text description
• Button controls that have no name available for screen readers

You might be surprised both at how many accessibility problems this scanner...

Additional ways to verify your code

We’ve covered some of the most common ways to verify your code. GitLab offers even more features that help you test your code further. We don’t have enough space to cover all of them in detail, but here’s a quick description of three additional methods you can use to test code. Details for enabling and configuring all of these tools are available in the official GitLab documentation.

Code coverage

Automated functional tests make sure that your code is doing what it’s supposed to do. Having tests in place is a critical part of every software development project, but it’s easy to get a false sense of confidence from seeing that all your tests are passing if you don’t know how much of your code base those tests cover. After all, having 100 passing tests doesn’t do you much good if all of those tests execute the same 5% of your application’s code.

Code coverage reports give you confidence...

Summary

Once again, you covered a lot of ground in this chapter. You saw how to build code within a GitLab CI/CD pipeline, using a variety of different methods and languages. This doesn’t cover every possible way you could compile or otherwise build your code—we’ve barely scratched the surface of that topic—but you should have a good idea of the general steps involved regardless of what language or tools you use. You also learned that certain kinds of code verification tools require that you build your code first because they interact with your code as it runs. Other tests don’t require this step because they simply scan your source code without running it.

Next, you saw how to use GitLab’s Code Quality feature within your pipelines to make sure your code follows best practices for coding style, adheres to common coding conventions, avoids unnecessary complexity, and doesn’t exhibit any code smells that indicate the possible presence...