Chapter 5: Running Quality Tests in a Build Pipeline

In the previous chapter, we introduced Azure Pipelines and learned how to implement a CI/CD process using Azure DevOps, GitHub, and containers.

In this chapter, we are going to cover how to run quality tests in a build pipeline. We will begin by explaining what the benefits of automatic testing are. Then, we will look at how to run unit tests in a build pipeline, how to perform code coverage testing, and how to view the test results. Finally, we will cover how to use Feature Flags to test code in production.

The following topics will be covered in this chapter:

• Benefits of automatic testing
• Introduction to unit testing
• Running unit tests in a build pipeline
• Introduction to code coverage testing
• Performing code coverage testing
• Assigning test results to work items
• Introduction to Feature Flags
• Using Feature Flags to test in production

Technical requirements

To follow this chapter, you need to have an active Azure DevOps organization. The organization that will be used in this chapter is called the Parts Unlimited organization. It was created in Chapter 1, Azure DevOps Overview. You also need to have Visual Studio 2019 installed, which can be downloaded from https://visualstudio.microsoft.com/downloads/. For the latest demo, you will need Visual Studio Code with the C# extension installed and the .NET Core SDK, version 3.1 or later.

The source code for our sample application can be downloaded from the following link:

https://github.com/PacktPublishing/Learning-Azure-DevOps---B16392/tree/master/Chapter%205/RazorFeatureFlags

Benefits of automatic testing

After adding a new feature to your application, you want to know if it will work correctly, given all the possible interactions. You also don't want to break any other features with this new functionality and want to know if the code is easily understood by others, as well as being maintainable.

All of this can be tested manually. But as the project and code base grows over time, testing all these features manually can become repetitive and error-prone. This is where automatic testing comes in.

There has always been a great deal of controversy around automatic testing. Many people believe that testing is too expensive to create and maintain. This is indeed true when tests are created badly. But when automatic tests are created properly, it will absolutely lower the amount of time and costs compared to frequent manual testing or releasing poor-quality software. Using automatic testing, your project will benefit from the ability to release software...

Introduction to unit testing

With unit testing, you break up code into small pieces, called units, that can be tested independently from each other. These units can consist of classes, methods, or single lines of code. The smaller the better works best here. This will give you a better view of how your code is performing and allows tests to be run fast.

In most cases, unit tests are written by the developer that writes the code. There are two different ways of writing unit tests: before you write the actual production code, or after. Most programmers write it afterwards, which is the traditional way of doing things, but if you are using test-driven development (TDD), you will typically write them beforehand. Unit testing will also make code documentation easier. It encourages better coding practices and you can leave code pieces to describe the code's functionality behind. Here, you will focus more on updating a system of checks.

In the next section, we are going to cover...

Running unit tests in a build pipeline

Our Parts Unlimited test project already has unit tests created. So, this is a good pick for this demo. First, we are going to look at the application and the tests that are created. Therefore, we have to clone the repository to our local filesystem and open the solution in Visual Studio.

Downloading the source code

We are going to create unit tests for the web application for Parts Unlimited. First, we need to clone the repository from Azure DevOps to our filesystem. This will allow us to add the unit tests to it using Visual Studio Code. Therefore, we must take the following steps:

1. Open a web browser and navigate to https://dev.azure.com/.
2. Log in with your Microsoft account and select the Parts.Unlimited project. Then, from the left menu, select Repos. This will let you navigate to the source code of the project.
3. From the top-right menu, select Clone:

   Figure 5.1 – Search result

4. On the next screen, make sure...

Introduction to code coverage testing

With code coverage testing, you can measure what source code for an application is going to be tested. Code coverage testing measures how many lines, blocks, and classes are executed while automated tests, such as unit tests, are running.

The more code that's tested, the more confident teams can be about their code changes. By reviewing the outcome of the code coverage tests, teams can identify what code is not covered by these tests. This information is very helpful as it reduces test debt over time.

Azure DevOps supports code coverage testing from the build pipeline. The Test Assemblies task allows us to collect code coverage testing results. There is also a separate task, called Publish Code Coverage Results, that can also publish these results. This task offers out-of-the-box support for popular coverage results formats such as Cobertura and JaCoCo.

Important Note

Cobertura and JaCoCo are both Java tools that calculate the...

Performing code coverage testing

To perform code coverage testing, we need to open the build pipeline that we created in the previous demo. Let's get started:

1. With the build pipeline open, select the Edit button in the right-hand corner:

   Figure 5.16 – Editing the pipeline from the previous demo

2. Navigate to the Test Assemblies task to open the settings.

   Under Execution settings, check the Code coverage enabled box:

   Figure 5.17 – Enabling code coverage testing

3. Now, Save and queue the build, specify a save comment, and wait until the pipeline is fully executed. The Visual Studio Test task creates an artifact that contains .coverage files that can be downloaded and used for further analysis in Visual Studio.
4. After executing the pipeline, on the overview page of the build, select Code Coverage from the top menu and click on Download code coverage results. A file with the .coverage extension will be downloaded to your local filesystem.
5. Double-click...

Assigning test results to work items

Once the test has run automatically and the build process has finished, you can assign the results to work items that have been added to the backlog and sprint. For this, you must perform the following steps:

1. Go back to the build pipeline and select the pipeline that ran last. Click Test from the top menu.
2. For the results table, make sure that Passed is selected and that Failed and Aborted have been deselected:

   Figure 5.19 – Selecting all the passed tests

3. Then, select a couple of tests. After doing this, from the top menu, click Link:

   Figure 5.20 – Linking the items

4. Search for As a tester and select the work item that is displayed as a search result:

   Figure 5.21 – Selecting the work item

5. Click Associate to link the work item to the test result.
6. Now, click on one of the test results that's linked to the work item. This will show the details for this item. From here, you can click on work items from...

Introduction to Feature Flags

You can use a Feature Flag to turn features in your code, such as specific methods or sections in your code, on or off. This can be extremely helpful when you want to hide (disable) and expose (enable) features in a solution. Features that are not complete and ready for release yet can be hidden or exposed in the solution. This allows us to test code in production for a subset of users. You can enable the code for a subset of users, for instance, based on the login name of the user and let them test the features before releasing them to others. However, there is a drawback to Feature Flags: they introduce more complexity in your code, so it is better to constrain the number of toggles in your application.

The recommended approach when creating Feature Flags is to keep them outside the application. For instance, a web or app configuration file is a good place to add Feature Flags because you can change them easily, without the need to redeploy the application...

Using Feature Flags to test in production

In this demonstration, we are going to create a new .NET Core application in Visual Studio Code. Then, we are going to implement a Feature Flag for this application.

We are going to add a very basic Feature Flag that changes the welcome message from Welcome to Welcome to Learn Azure DevOps. This is only going to be tested by a subset of users. Therefore, we need to open Visual Studio Code and create a new Razor application with .NET Core. I have created a new folder on my local filesystem called FeatureFlags for this. Open this folder in Visual Studio Code. Check the next section for the detailed steps.

Creating a new .NET Core application

To create a new .NET Core application, follow these steps:

1. With Visual Studio Code open, click on Terminal > New terminal from the top menu.
2. In the Terminal, add the following line of code to create a new project:

   dotnet new webapp -o RazorFeatureFlags
   code -r RazorFeatureFlags

...

Summary

In this chapter, we covered how to run quality tests in a build pipeline in more depth. With this, you can now run unit tests from the build pipeline and execute coverage tests from Azure DevOps. Lastly, we covered how to create Future Flags inside an application that you can use in your future projects as well.

In the next chapter, we are going to focus on how to host build agents in Azure Pipelines.

Further reading

Check out the following links for more information about the topics that were covered in this chapter:

• Unit test basics: https://docs.microsoft.com/en-us/visualstudio/test/unit-test-basics?view=vs-2019
• Run quality tests in your build pipeline by using Azure Pipelines: https://docs.microsoft.com/en-us/learn/modules/run-quality-tests-build-pipeline/
• Explore how to progressively expose your features in production for some or all users: https://docs.microsoft.com/en-us/azure/devops/migrate/phase-features-with-feature-flags?view=azure-devops