Chapter 3: Architecting for DevOps Quality

The overall aim of DevOps is to deliver high performance and quality to IT projects. In this chapter, you will learn how DevOps can add value to the quality of IT delivery. In this chapter, we will learn how to define test strategies, proving that quality has been delivered according to the Definition of Done (DOD). But what happens if something breaks? The golden rule in DevOps is you build it, run it, break it, and then you fix it. But then, we must detect what the issue is by executing a root cause analysis (RCA). In the final section, we will discuss remediation and, with that, continuous improvement.

After completing this chapter, you will be able to identify and implement quality measures in DevOps projects. You will have learned what tests can be included, how the are organized, and what the value of these tests is, thus continuously improving the product or service.

In this chapter, we're going to cover the following main...

Defining test strategies

In the previous chapter, we concluded that testing is a crucial step in the CI/CD process to ensure the quality of the build. In this section, we will learn how to define test strategies in DevOps.

First, DevOps requires a different approach to testing: it's part of the continuous deployment and integration of builds. The reason we should adopt DevOps is because enterprises want to speed up releases so that they can act much quicker to changing demands. For testing, this means that there is a shift from testing the end product to continuous testing, with a focus on reducing build and test times.

In other words, testing is no longer just a matter of detecting the faults in the end product; it has become part of the full life cycle of the build, collecting feedback during this whole cycle.

Testers should be members of the DevOps team. Their responsibility is to constantly collect feedback, measure the cycle time, and find ways to reduce these...

Implementing quality measures

By now, it should be clear that everything in DevOps is about being continuous, which, in other words, means continuous deployment, continuous integration, continuous testing, and continuous quality engineering. DevOps projects constantly focus on quality at every stage of development and operations. It's different from traditional approaches where teams have a separate phase to fix issues. In DevOps, teams constantly measure the products and fix issues as soon as they occur. One of the six DevOps principles is continuous improvement, which refers to the feedback loop wherein products are improved in every iteration, but also to the DevOps process itself.

A common practice in IT projects was to have a fixing phase, something that Gerald Weinberg describes in his book Perfect Software and other illusions about testing. The fixing phase was put at the end of the development phase before software was handed over to operations. In DevOps, we don&apos...

Designing test automation and execution

In this section, we will learn how to design and implement tests. We will study the most common different test varieties and learn where we can use them. When we start discussing testing in IT, we need to discuss and agree upon a test management approach. In this book, we will use TMAP, introduced by ICT service provider Sogeti in 1995 and widely accepted as the standard in software testing.

The traditional phases of TMAP are as follows:

• Planning
• Preparation
• Specification
• Execution
• Evaluation

In DevOps, this is not a one-off exercise; we will be working with continuous testing. One major difference with the traditional way of working is that there's no separate test unit with a manager and testers. These professionals are now part of the DevOps team and they do their work alongside the developers. Next, in DevOps, we are working according to the everything as code principle, thus allowing teams to automate...

Understanding root cause analysis

In the previous sections, we discussed quality measures and testing to validate these criteria in a highly structured and automated way. Still, things can go wrong. The golden rule in DevOps is you build it, you run it, often followed by the statement you break it, you fix it. Or even it could be you destroy it, you rebuild it better. If something breaks, the team will need to find out what exactly happened. In this section, we will talk about root cause analysis (RCA) as one of the most important instruments for finding the cause of a problem.

RCA is the methodology for finding the exact cause of an issue. With that, RCA provides insights on how the team can improve products or services. These can be quick fixes or long-term enhancements. RCA is more than just a way to find problems; it's the start of improvement. Important questions that need to be addressed in RCA are as follows:

• What is the problem?
• Where was it found?
...

Designing for remediation

So far, we've talked about coding the software, implementing the required infrastructure, automating it all through CI/CD pipelines, testing the environments, detecting issues, and, if needed, fixing the problems. But there's something that we haven't been discussing yet and that's the speed of software development and DevOps itself.

DevOps is about learning. As the team and projects grow, they learn how to improve. They learn from the product itself and how it's used, and they learn from looking at other projects, technologies, and methodologies. These lessons are adopted and injected into their own project. The team doesn't need to start over, though – they can adopt and adapt as they proceed. We call this remediation, which is the process of improving an existing situation.

Remediation can take place on three levels, as follows:

• Infrastructure: Assuming that we build everyone according to the "...

Summary

This chapter was all about quality. We learned how to identify quality measures and that this is more than just about fixing bugs. Quality is about meeting expectations, but DevOps teams need to be able to measure these expectations. That's why businesses, developers, and operators need to be clear on the acceptance criteria. In this chapter, we discussed the DoR as an entry point to working on a project and DoD for measuring whether a product is really complete.

Measuring means that teams have to test. In a traditional way of working, testing is done as soon as the whole product is delivered. In DevOps, we work with continuous testing. In other words, all the team members are involved in testing and validating the quality of the product. In this chapter, we discussed different ways and types of testing that are common in DevOps. Lastly, we talked about continuous improvement using remediation. Cloud platforms, software development technology, and DevOps tools are...

Questions

1. In this chapter, we discussed different types of tests. One of them is unit tests. Give a short description of a unit test.
2. In a data-oriented test, we enter the minimum and the maximum values. If we enter a value within these boundaries, the test result should be valid. What is this test method called?
3. To decide if a product is complete, DevOps uses a certain technique. What is this technique called?
4. True or False: A fishbone diagram is a good practice for analyzing the root cause of a problem.

Further reading

• Quality for DevOps Teams, by Rik Marselis, Berend van Veenendaal, Dennis Geurts and Wouter Ruigrok, Sogeti, 2020
• Test-Driven Development: By Example, by Kent Beck, 2002