Moving to Continuous Integration

After setting up source control for your organization and deciding on a branching and merging strategy that supports parallel work, you are ready to move on to continuous integration. Continuous integration is a method where every developer takes their work and integrates it with the work of others, and then verifies the quality of the combined work. The value of this is an increase in quality early on in the pipeline. This reduces the risk of error later on when merging code changes and reduces the number of bugs that are found in production, thereby reducing costs and protecting your reputation.

Continuous integration is only possible when you have the proper setup with the necessary tools. In this chapter, you will learn how to use Azure DevOps pipelines to set up continuous integration.

The following topics will be covered in this chapter:

...

Technical requirements

To go through the recipes that are covered in this chapter, you will need an Azure DevOps organization.

Introducing continuous integration

Continuous integration is a methodology where you integrate your own changes with those of all of the other developers in your project and test whether the combined code still works as expected. This way, you create a fast loop that provides you with feedback on your work.

When working with extensive branching strategies for isolating code changes, it is not uncommon for one or more developers to work for days, weeks, or even months on an isolated branch. While this is great for making sure that their changes do not disrupt others, it is also a great way to make sure that there won't be merge issues later. If you have ever had to merge weeks or months of work back into a master branch, you will know how much work is involved and how often this results in bugs or other issues.

To prevent this, developers should make it a habit to integrate...

Creating a build definition in Azure DevOps

The main way to perform continuous integration is by using a continuous integration build. In Azure DevOps, builds can be configured as part of the Azure Pipelines offering. There are currently two approaches available for creating a build definition:

• Via the visual designer (also called classic builds and releases)
• Through Yet Another Markup Language (YAML) files (also called YAML pipelines or multistage pipelines)

The rest of this section focuses on the visual designer. The following section, YAML build definitions, will go into more detail about YAML pipelines. Both approaches support roughly the same capabilities, although there are some differences. Some features that are available in classic builds and releases are not (yet) available in YAML build definitions. Also, some new features are only provided to YAML pipelines.

If you...

Running a build

In this section, you will learn how to work with the build results and use them to report and generate builds. You will also learn how to run a build with every pull request and report the quality of the changes back to that pull request to assist the reviewer.

Viewing the build results

While a build is running, an agent will perform all the configured steps one by one. Azure Pipelines will capture detailed information and logs from all these steps. As you can see in the following screenshot, a build will display a list of all the steps it has executed on the left. Clicking on any of these steps will open a detailed view that displays the logs per step:

Whenever there are warnings or errors during the build...

Working with YAML pipelines

You have seen how to create a build definition using the visual designer. A new, alternative approach, which has been available since early 2019, is the use of YAML pipelines. When working with YAML pipelines, you specify your complete build definition in a YAML file and store it in source control, often next to the source code that the build is for.

While both pipeline systems coexist, using YAML pipelines is now the preferred approach for defining pipelines. This means that it is very likely that new features will only surface in YAML pipelines.

The reason for using build definitions as code

When you first start working with YAML build definitions, you might find that the learning curve is steeper...

Agents and agent queues

The build definitions that you have created so far can contain agent jobs, which in turn contain tasks. These tasks are not executed within your Azure DevOps organization directly, but are executed by agents that run on VMs or in containers. In turn, agents are grouped in agent pools. There are two types of agent pools that you can work with:

• Built-in agent pools
• Self-hosted agent pools

Let's go through them one by one.

Built-in agent pools

Built-in agent pools are managed by Microsoft and are made available to you as part of the product. There are different agent pools available, depending on your needs. Pools run different versions of Windows and Visual Studio, and there are also pools available...

Other tools

There are many tools available besides Azure DevOps. Two other well-known tools are GitLab CI and Jenkins. Some very basic knowledge of these tools will help you to understand how to integrate with them if that is ever necessary. Also, a limited understanding of other tools will help you to more quickly understand the concepts and generalize your knowledge of how to work with these other tools.

To highlight how these tools work with the same concepts, both examples in this section are equivalent to the Azure DevOps YAML pipeline in the Writing a YAML build definition section.

GitLab CI

GitLab offers build pipelines using the GitLab CI capabilities. GitLab CI is configured by putting a file with the .gitlab-ci.yml...

Summary

In this chapter, we looked at continuous integration and learned how it is a combination of your mindset, the process, and tools. You learned how to create build definitions using Azure Pipelines using both the graphical designer and YAML, as well as how to run builds. You learned that you can use build pipelines to compile and test your code, as well as report the outcome back to pull requests.

You learned that builds can produce outcomes, called artifacts. Artifacts are stored and retained within Azure pipelines and can be used to store reports, but are also the starting point of deployment pipelines, which you will learn about in the next chapter. You also learned about the infrastructure that you need to run builds—namely, agents and agent pools. Finally, you saw two brief examples of how to run a continuous integration build using GitLab CI and Jenkins, which...

Questions

As we conclude, here is a list of questions for you to test your knowledge regarding this chapter's material. You will find the answers in the Assessments section of the appendix:

1. True or false – you achieve continuous integration if you compile all the branches of your project at least daily.
   
2. True or false – a classic build definition is always connected to a source code repository.
3. True or false – a YAML pipeline definition is always connected to a source code repository.
4. Which of the following is needed to call an external tool from an Azure pipeline?
   1. An external service definition
   2. An Azure services connection
   3. A service connection
   4. A service locator

5. What are some common reasons for using self-hosted agents? (Choose all of the correct answers from the following:)
   1. Access to closed networks is needed.
   2. Specific extension tasks need to...

Further reading

• An in-depth definition of continuous integration by Martin Fowler is available at https://martinfowler.com/articles/continuousIntegration.html.
• A detailed description of the conditions syntax is available at https://docs.microsoft.com/en-us/azure/devops/pipelines/process/conditions?view=azure-devops&tabs=classic.
• Exercises for practicing with Azure DevOps builds can be found at https://docs.microsoft.com/en-us/learn/modules/create-a-build-pipeline/index.
• You can find the Visual Studio marketplace for Azure DevOps at https://marketplace.visualstudio.com/azuredevops.
• You can find a detailed description of the Azure Pipelines YAML syntax at https://docs.microsoft.com/en-us/azure/devops/pipelines/yaml-schema?view=azure-devops&tabs=schema.
• Details of the pricing of the Azure pipelines hosted and self-hosted agent pools are available at https://azure.microsoft...