Home Cloud & Networking Designing and Implementing Microsoft DevOps Solutions AZ-400 Exam Guide - Second Edition

Designing and Implementing Microsoft DevOps Solutions AZ-400 Exam Guide - Second Edition

By Subhajit Chatterjee , Swapneel Deshpande , Henry Been and 1 more
ai-assist-svg-icon Book + AI Assistant
eBook + AI Assistant $37.99 $25.99
Print $46.99 $27.99
Subscription $15.99 $10 p/m for three months
ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription.
ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription. $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime! ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription.
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Along with your eBook purchase, enjoy AI Assistant (beta) access in our online reader for a personalized, interactive reading experience.
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription. ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription. BUY NOW $10 p/m for first 3 months. $15.99 p/m after that. Cancel Anytime! ai-assist-svg-icon NEW: AI Assistant (beta) Available with eBook, Print, and Subscription.
eBook + AI Assistant $37.99 $25.99
Print $46.99 $27.99
Subscription $15.99 $10 p/m for three months
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 7000+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook + Subscription?
Download this book in EPUB and PDF formats, plus a monthly download credit
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with a Packt Subscription?
Gain access to our AI Assistant (beta) for an exclusive selection of 500 books, available during your subscription period. Enjoy a personalized, interactive, and narrative experience to engage with the book content on a deeper level.
This book & 6500+ ebooks & video courses on 1000+ technologies
60+ curated reading lists for various learning paths
50+ new titles added every month on new and emerging tech
Early Access to eBooks as they are being written
Personalised content suggestions
Customised display settings for better reading experience
50+ new titles added every month on new and emerging tech
Playlists, Notes and Bookmarks to easily manage your learning
Mobile App with offline access
What do you get with eBook?
Along with your eBook purchase, enjoy AI Assistant (beta) access in our online reader for a personalized, interactive reading experience.
Download this book in EPUB and PDF formats
Access this title in our online reader
DRM FREE - Read whenever, wherever and however you want
Online reader with customised display settings for better reading experience
What do you get with video?
Download this video in MP4 format
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with video?
Stream this video
Access this title in our online reader
DRM FREE - Watch whenever, wherever and however you want
Online reader with customised display settings for better learning experience
What do you get with Audiobook?
Download a zip folder consisting of audio files (in MP3 Format) along with supplementary PDF
What do you get with Exam Trainer?
Flashcards, Mock exams, Exam Tips, Practice Questions
Access these resources with our interactive certification platform
Mobile compatible-Practice whenever, wherever, however you want
  1. Free Chapter
    Chapter 1: Introduction to DevOps
About this book
The AZ-400 Designing and Implementing Microsoft DevOps Solutions certification helps DevOps engineers and administrators get to grips with practices such as continuous integration and continuous delivery (CI/CD), containerization, and zero downtime deployments using Azure DevOps Services. This new edition is updated with advanced topics such as site reliability engineering (SRE), continuous improvement, and planning your cloud transformation journey. The book begins with the basics of CI/CD and automated deployments, and then moves ahead to show you how to apply configuration management and Infrastructure as Code (IaC) along with managing databases in DevOps scenarios. As you make progress, you’ll explore fitting security and compliance with DevOps and find out how to instrument applications and gather metrics to understand application usage and user behavior. This book will also help you implement a container build strategy and manage Azure Kubernetes Services. Lastly, you’ll discover quick tips and tricks to confidently apply effective DevOps practices and learn to create your own Azure DevOps organization. By the end of this DevOps book, you'll have gained the knowledge needed to ensure seamless application deployments and business continuity.
Publication date:
September 2022
Publisher
Packt
Pages
490
ISBN
9781803240664

 

Introduction to DevOps

DevOps is not a product or tool that you can buy or install. DevOps is about culture and the way you write, release, and operate your software. DevOps is about shortening the time between a new idea and your first end user experiencing the value it delivers. In this book, you will learn about the tools and techniques you can use to apply that philosophy to your way of working. As the purpose of this book is to help you prepare for the AZ-400 certification exam, the core concepts and DevOps-related practices will be illustrated using Azure DevOps and its associated technologies.

DevOps has gained popularity in recent years, with almost all software engineering teams adapting themselves to a new world of doing things using a newer set of tools that activate productivity and better collaboration across development and operations. DevOps is often seen as an extension of Agile, but its scope is much wider, wherein it complements Agile.

In this first chapter, you will learn more about what DevOps is and how to recognize a successful DevOps team. By the end of this chapter, you will be familiar with the key terms and be equipped with a broader understanding of the high-level principles and practices of DevOps. You will also discover the benefits of DevOps as it applies to software engineering teams.

In this chapter, we will cover the following topics:

  • What is DevOps?
  • Creating your ideal DevOps organization
  • Exploring DevOps practices and habits
  • The five stages of the DevOps evolution
 

Technical requirements

There are no technical requirements for this chapter.

 

What is DevOps?

If you were to list all of the different definitions and descriptions of DevOps, there would be many. However, as different as these might be, they most likely share several concepts. These are collaboration, continuous delivery of business value, and breaking down silos.

With all the technical discussion in the rest of this book, it is important not to overlook the value proposition for adopting DevOps – namely, that it will help you improve the way that you continuously deliver value to your end users. To do this, you must decrease the time between starting work on a new feature and the first user using it in production. This means that you not only have to write the software but also deliver and operate it.

Over the last decade, the way we write software has fundamentally changed. More and more companies are now adopting an agile way of working to increase the efficiency of their software development. More and more teams are now working in short iterations or sprints to create new increments of a product in quick succession. However, creating potentially shippable increments faster and faster does not create any value by itself. Only when each new version of your software is also released to production and used by your end users does it start delivering value.

In traditional organizations, developers and operators are often located in different departments, and taking software into production includes a hand-off, often with a formal ceremony around it. In such an organization, it can be hard to accelerate that delivery to production, along with the speed at which development can create new versions.

Next to that, the development and operations departments often have conflicting goals. While a development department is rewarded for creating many changes as fast as possible, operations departments are rewarded for limiting downtime and preventing issues. The latter is often best achieved by making as few changes as possible. The conflict here is clear – both departments have optimizations for one subgoal, as shown in the following diagram:

Figure 1.1 – Conflicting goals between development and operations

Figure 1.1 – Conflicting goals between development and operations

This defeats the purpose of these subgoals, which comes from the shared, overarching goal of quickly taking in new versions while maintaining stability. It’s precisely this conflict between developmental and operational goals that is one of the things that should disappear in a DevOps culture. In such a culture, development and operations teams should work together on delivering new versions to production in a fast and reliable manner and share responsibility for both subgoals.

While it is good to know that DevOps is a cultural movement, tools and automation are an important part of that culture. In this book, we will focus on these tools and how to use them to implement many of the practices that come with a DevOps culture. In other words, this book will be mostly about the products and processes associated with DevOps. If you want to learn more about the cultural side of things and the people, there are many other books you can read. A very good read is The Phoenix Project: A Novel About IT, DevOps, And Helping Your Business Win, by Gene Kim.

The rest of this section will explore the relationship between DevOps and Agile to see how they complement each other. The focus will be on agile techniques and prices for work management. We will also discuss the goals and benefits of a DevOps culture.

The relationship between DevOps and Agile

If you take a look at Agile, you may notice that part of it focuses on business value and shortening the time of delivering a new business value. From that perspective, adopting DevOps is a logical next step after Agile. Agile advocates that the software development teams’ responsibilities should extend forward by engaging with users and other stakeholders to deliver valuable and potentially shippable products quicker. DevOps is not just about something that might be shipped, but shipping it as well. With Agile and DevOps combined, you can create an end-to-end and continuous flow of value to your users.

You will need a common approach to managing the work to be done for everyone involved. In the next section, you will find some pointers on how to incorporate operational concerns in the way you manage your work.

Agile work management

When you are starting to increase the collaboration between development and operations, you will quickly notice that they have to cope with different types of work. In development, a large part of the work is planned: user stories and bugs that are picked up from a backlog. On the other hand, for operations, a large part of their work is unplanned. They respond to warnings and alerts from systems and requests or tickets from users or developers.

Integrating these two, especially if developers and operators are located on the same team, can be challenging. To learn how to deal with this, let’s explore the following approach:

  1. First, switch to a flow-based way of working for developers.
  2. Next, allow for operations to also list their work in the same work management system as developers using synchronizations. You can also choose to implement fastlaning, a way to expedite urgent work.
  3. Finally, you may choose to decommission existing ticketing tools for operations if possible.

Fastlaning is an approach to organizing work that allows for both planned and unplanned work by visualizing two separate lanes of work. To do this, the Scrum board is extended with a Kanban-like board on the top. This is the fast lane. On the Kanban board, urgent but unplanned work is added. Any work that’s added to this lane is picked up by the team with the highest priority. Only when there is no work remaining in the fast lane is work from the Scrum board, along with planned work, picked up. Whenever new work is added to the fast lane, this takes priority again. Often, there is the agreement that any work in progress is finished before switching to work in the fast lane.

Important Note

Dependency management is also an important aspect of agile work planning. Hence, teams often make use of the prioritization attribute to qualify more important work for the short term.

Switching to a flow-based methodology

The first thing to consider when switching to a flow-based methodology is transitioning the way developers work from batch-wise to flow-based. An example of a batch-wise way of working is Scrum. If you are using the Scrum framework, you are used to picking up a batch of work every 2 to 4 weeks and focusing on completing all of that work within that time window. Only when that batch is done do you deliver a potentially shippable product.

When changing to a flow-based approach, you try to focus not on a batch, but just on one thing. You work on that one work item and drive it completely until it’s done before you start on the next. This way, there is no longer a sprint backlog, only a product backlog. The advantage of this approach is that you no longer decide which work to perform upfront; whenever you are free to start on new work, you can pick up the next item from the backlog. In an environment where priorities shift quickly, this allows you to react to change quickly.

These changes to the way developers organize their work make it easier to include operations in work management, but there is also another benefit. When developers are focusing on getting a single work item done instead of a whole sprint at once, you can also increase the number of times you can deliver a small portion of value to your users.

Synchronizing work items to one system

Once the development team has changed the way it organizes its work, it should be easier for developers to also list their planned work on the shared backlog and pull work from that backlog when they have time to work on it. They now also have a place where they can list their unplanned work.

However, there may still be an existing ticketing system where requests for operations are dropped by users or automatically created by monitoring tools. While Azure DevOps has a great API to rework this integration to directly create work items in Azure DevOps, you may first choose to create a synchronization between your existing ticketing tool and Azure Boards. There are many integration options available and there is a lot of ongoing work in this area. This way, operators can slowly move from their old tool to the new one, since they are now in sync. Of course, the goal is for them to move over to the same tool, as the developers, completely.

Fastlaning

With the work of developers and operators in the same work management tool, you will notice that you have a mix of planned and unplanned, often urgent, work in the system. To ensure that urgent work gets the attention and precedence it deserves, you can introduce what is called a fast lane to your sprint board. The following screenshot shows an example of an Azure board that has been set up for fastlaning production issues:

Figure 1.2 – Azure Board setup depicting the fast lane

Figure 1.2 – Azure Board setup depicting the fast lane

The horizontal split in this board is only used to work on tasks in the regular lane when there is no work to be picked up in the fast lane.

You can find instructions on how to configure swim lanes in your Azure (Kanban) boards for expediting work at https://docs.microsoft.com/en-us/azure/devops/boards/boards/expedite-work?view=azure-devops.

Decommissioning other work management tools

After creating a shared work management system between development and operations, there is an opportunity to increase the amount of collaboration between them. When this collaboration is taking off, old ticketing systems that were used by operations may now slowly be decommissioned over time. Integrations from monitoring tools can be shifted to the new shared tools, and the number of tickets between developers and operators should slowly decrease as they find new ways of working together.

Important Note

Azure DevOps allows you to customize work item templates, as well as define life cycle states. Using this feature, teams can easily model their work item template types based on any existing taxonomy they might be using in their existing tools. This significantly reduces the learning curve in the adoption of the new shared work management tool. For more information on this, go to https://docs.microsoft.com/en-us/azure/devops/boards/backlogs/work-item-template?view=azure-devops&tabs=browser#manage-work-item-templates.

Goals and benefits of a DevOps culture

At this point, you might be wondering about the point of it all. What are the benefits of DevOps and what’s in it for you, your colleagues, and your organization? The most common goal of adopting DevOps is to achieve a reduction in cycle time. Cycle time is the time between starting work on a new feature and the moment that the first user can use it. The way this is achieved, by automation, also serves the goals of lower change failure rate, lower mean time to repair (MTTR), and lower planned downtime.

Next to all that, there may be other benefits, such as increased employee satisfaction, less burnout and stress, and better employee retention. This is attributed to removing opposing goals between developers and operators.

For a while, there was doubt about whether DevOps works, whether these goals can be met, and whether the extra benefits can be achieved since this was only shown using case studies. The downside of this is that case studies are often only available for successful cases, not for unsuccessful ones. This all changed in 2018 when the book Accelerate came out. This book shows, based on years of quantitative research, that modern development practices such as DevOps contribute to reaching IT goals and organizational goals.

Measuring results

To measure where you currently stand as a team or organization and the impact DevOps has on you, there are several metrics that you can start recording. As always, when working with metrics or key performance indicators (KPIs), make sure that you do not encourage people to game the system by looking only at the numbers. Several interesting metrics are detailed in the following sections and if you go over them, you will notice that they are all about encouraging flow.

Cycle time and lead time

Cycle time and lead time are metrics that come from Lean and Kanban and are used to measure the time needed to realize a change. Cycle time is the amount of time between starting work on a feature and users being able to use that feature in production. The lower the cycle time, the quicker you can react to changing requirements or insights. Lead time is the amount of time between requesting a feature and realizing that feature. It is the time between adding work to the backlog and when you start implementing it.

When you add cycle time and lead time together, you are calculating another metric, known as the time to market. This is often an important business metric when developing software. Hence, minimizing both cycle time and lead time will have a business impact.

The amount of work in progress

Another thing you can measure is the amount of work in progress at any point in time. DevOps focuses on the flow of value to the user. This implies that everyone should, if possible, be doing only one thing at a time and finish that before moving on to something else. This reduces the amount of time spent on task switching and the amount of time spent on not yet complete work. Measuring how many things a team works on in parallel and reporting on this can act as a source of encouragement.

You can even go as far as putting actual limits on the amount of work that can be in progress. The following is a small part of Figure 1.2, showing that these work-in-progress limits can even be shown in the tool:

Figure 1.3 – Azure Boards depicting limits for each stage

Figure 1.3 – Azure Boards depicting limits for each stage

The goal is to have as little work in progress at the same time as possible.

Mean time to recovery

The third metric is the mean time to recovery. How long does it take you to restore a service in case of a (partial) outage? In the past, companies focused on reducing the mean time between failures. This used to be the mean indicator of the stability of a product. However, this metric encourages limiting the number of changes going to production. The unwanted consequence is often that outages, though they might be rare, last long and are hard to fix.

Measuring the mean time to recovery shifts the attention to how quickly you can remediate an outage. If you can fix outages quickly, you can achieve the same – namely, you can minimize the amount of downtime without sacrificing the rate of change. The goal is to minimize the time to recovery.

Change rate and change failure rate

Finally, you can measure the number of changes that are delivered to production and the percentage of that which is not successful. Increasing the rate of change implies that you are delivering value to your users more often, hence realizing a flow of value. Also, by measuring not just the number of failures but also the percentage that fails, you are encouraging many small, successful changes instead of encouraging whether the number of changes is limited overall.

Your goal should be to increase the rate of change while lowering the change failure rate. Apart from the four major KPIs listed in this section, many other metrics may be useful in measuring your DevOps maturity. All these metrics must be linked back to the important business objectives and key results (OKRs) that are expected. You can find more information about OKRs here: https://docs.microsoft.com/en-us/azure/cloud-adoption-framework/strategy/business-outcomes/okr.

A representative sample, for illustration purposes, is depicted in the following table:

Objective

Key Results

Faster time to market

  • Deployment Frequency: Every week
  • Deployment Time <= 4 hours
  • Lead Time (Major Releases): Once every quarter

Increase the business value that’s been realized while maintaining or reducing costs

  • CI/CD processes: 100% automated
  • Resource Utilization (95th percentile): 80%
  • Dashboards for monitoring both Health and Costs

Predictable and quality delivery and faster correction with fewer defects

  • High Availability > 99.9%
  • RTO < 1 hour, RPO < 15 mins

Better processes across IT, automation, teamwork, and culture

  • MTTR < 1 hour
  • Lead Time (Bugs) < 8 hour
  • Scaled Agile: Feature Teams > 5
  • Technical Debt < 1 week

Improved customer engagement and ability to quickly respond to market demands

  • CSAT: 4 or above
  • Product Planning: 50% of the backlog focuses on Customer Feedback

Table 1.1 – Using the OKR approach for your DevOps maturity

At this point, you might be wondering, how do I help my organization foster this culture and reap all of these benefits? The next section will answer this.

 

Creating your ideal DevOps organization

Well, maybe your organizational structure does not have to change at all. DevOps must start with a cultural change: openness, empathy, and collaboration are values that need to be encouraged. But still, changing your organizational structure may help accelerate this.

Traditionally, developers and operators are often organized into disparate teams or even different departments – organized in teams with people that have similar skill sets and responsibilities. A common change in organizations is changing this structure by pivoting and organizing teams behind a common goal, a single product, or a group of features, for example.

Now, you will need teams with different skill sets and responsibilities, teams most likely with developers and operators. It is important to realize that forcing such a change upon these people may not be the best way forward. Often, it works best to start with changing the culture and encouraging cooperation – then, this organizational change may come about naturally.

Finally, it is important to recognize one anti-pattern at this point. Some companies are trying to implement DevOps by hiring specialized DevOps engineers and positioning them between development and operations, interacting with both. While this, at first, may seem like a good idea, this goes against the DevOps values. If you do this, you are not breaking silos down, but you are adding a third one. You are not decreasing the number of hand-offs, you are most likely increasing them. Also, collaboration between developers and operations is often not enhanced by separating them using another organizational structure, and you may not see any increase in value to your end users at all.

Now that you know what DevOps is and you have a clear understanding of how you can form a DevOps team, it is time to explore how to start achieving your goals.

 

Exploring DevOps practices and habits

Since you are not the first team going on this journey, you can learn from the experiences of those before you. One example is the Microsoft team that built Azure DevOps. Being in the rare position that they can use their product for developing another product, they have learned a great deal about what makes DevOps successful. From this, they have identified seven key DevOps practices and seven DevOps habits that many successful DevOps teams share:

DevOps Practices

DevOps Habits

Configuration management

Team autonomy and enterprise alignment

Release management

Rigorous management of technical debt

Continuous integration

Focus on the flow of customer value

Continuous deployment

Hypothesis-driven development

Infrastructure as Code

Evidence gathered in production

Test automation

Live-site culture

Application performance monitoring

Myanage infrastructure as a flexible resource

Table 1.2 – DevOps practices and habits

Now, it is important to realize that just copying the motions described here will not guarantee success. Just as with Agile, you will have to spend time to understand these practices and habits, where they come from, and how they contribute to a continuous flow of value to your end users.

Important Note

Microsoft Services has introduced a more comprehensive DevOps Dojo model that aims to categorize the maturity of your DevOps practices. It gives you a good structure on how you should prioritize the DevOps-related investments within your teams. You can read more about it here: https://docs.microsoft.com/en-us/learn/paths/devops-dojo-white-belt-foundation/.

The following sections explore all of these practices and habits in more detail. Keep these in the back of your mind while reading the rest of this book. While the rest of this book will mostly focus on the technical means of how to do things, do not forget that these are only means. The real value comes from your mindset and creating a culture that focuses on creating a continuous flow of value for your customers.

DevOps practices

This section discusses all seven DevOps practices in turn. As you will quickly see, they are highly related, and it is quite hard to practice one without the other. For example, test automation is highly related to continuous integration and continuous deployment.

Important Note

If you plan on taking the AZ-400 exam, mastering all of these practices and performing them using Azure DevOps will help you significantly.

Configuration management

Configuration management is about versioning the configuration of your application and the components it relies on, along with your application itself. Configuration is kept in source control and takes the form of, for example, JSON or YAML files that describe the desired configuration of your application. These files are the input for tools such as Ansible, Terraform, Puppet, or PowerShell DSC, which configure your environment and application. These tools are often invoked from a continuous deployment pipeline.

The desired state can also be reapplied at an interval, even if no changes have been made to the intended configuration. This way, it is ensured that the actual configuration stays correct and that manual changes are automatically revoked. We call this the prevention of configuration drift. Configuration drift occurs over time due to servers being added or removed over time, or manual, ad hoc interventions by administrators. Of course, this implies that intended updates to the configuration are done in source control and only applied using tools.

Configuration management or Configuration as Code (CaC) is highly related to Infrastructure as Code (IaC). The two are often intertwined and on some platforms, the difference between the two may even feel artificial. CaC will be discussed in detail in Chapter 8, Implement Infrastructure and Configuration as Code.

Release management

Release management is about being in control of which version of your software is deployed to which environment. Versions are often created using continuous integration and delivery pipelines. These versions, along with all of the configuration needed, are then stored as immutable artifacts in a repository. From here on, release management tools are used to plan and control how these versions are deployed to one or more environments. Examples of such controls include manual approvals and automated queries of open work and quality checks before allowing deployment to a new environment.

Release management is related to continuous deployment and focuses more on controlling the flow of versions through the continuous deployment pipeline. Chapter 8, Implement Infrastructure and Configuration as Code, will cover CaC as part of release management.

Continuous integration

Continuous integration is a practice where every developer integrates their work with that of the other developers in the team at least once a day and preferably more often. This means that every developer should push their work to the repository at least once a day. A continuous integration build verifies that their work compiles and that all the unit tests run. It is important to understand that this verification should not run only on the code that the developer is working on in isolation. The real value comes when the work is also integrated with the work of others.

When integrating changes often and fast, problems with merging changes are less frequent and if they occur, they are often less difficult to solve. In Chapter 4, Everything Starts with Source Control, you will learn more about how to set up your source control repositories to make this possible. In Chapter 5, Moving to Continuous Integration, you will learn about setting up a continuous integration build.

Continuous deployment

Continuous deployment is the practice of automatically deploying every new version of sufficient quality to production. When practicing continuous deployment, you have a fully automated pipeline that takes in every new version of your application (every commit), results in a new release, and starts deploying it to one or more environments. The first environment is often called test and the final environment is called production.

In this pipeline, multiple steps verify the quality of the software before letting it proceed to the next environment. If the quality is not sufficient, the release is aborted and will not propagate to the next environment. The premise behind this approach is that, in the pipeline, you try to prove that you cannot take the current version to the next environment. If you fail to prove so, you assume it is ready for further progression.

Only when a release has gone through all the environments in the pipeline is it deployed to production. Whenever a release cannot progress to the next environment, that release will be completely canceled. While you might be inclined to fix the reason for the failure and then restart deployment from the point where it failed, it is important not to do so. The changes you made at that point are not validated by all of the controls that the version has already passed through. The only way to validate the new version as a whole is by starting the pipeline from the start. You can see this clearly in the following diagram:

Figure 1.4 – Continuous deployment flow

Figure 1.4 – Continuous deployment flow

In Chapter 6, Implementing Continuous Deployment and Release Management, you will learn about setting up continuous deployment using.

Important Note

The preceding diagram can be found at https://en.wikipedia.org/wiki/Continuous_delivery#/media/File:Continuous_Delivery_process_diagram.svg. The image is by Grégoire Détrez, original by Jez Humble, under CC BY-SA 4.0, at https://creativecommons.org/licenses/by-sa/4.0/.

Infrastructure as Code

When writing an application, the binaries that you are building must be running somewhere on some application host. An example of such an application host can be a web server such as IIS or Apache. Next to an application host, we may need a database and some messaging solution. This is called the infrastructure for our application. When practicing IaC, you are keeping a description of this infrastructure in your source code repository, alongside your application code.

When the time comes to release a new version of the application and you need to make one or more changes to the infrastructure, you are executing this description of your desired infrastructure using tools such as Chef, Puppet, Terraform, Azure Bicep, PowerShell DSC, or Azure ARM templates. The execution of such a description is idempotent, which means that it can be executed more than once, and the result is the same. This is because your description of the infrastructure describes the desired state you want the infrastructure to be in and not a series of steps to be executed. Those steps to be executed, if there are any, are automatically determined by your tool of choice. Applying the desired state can also be done automatically in a continuous deployment pipeline and is often executed before the application code is updated.

The big advantage of this is that you can easily create a new environment where the infrastructure is guaranteed to be the same as in your other environments. Also, the problem of configuration drift, where the infrastructure between your different environments slowly diverges, is no longer possible since every time you apply the desired state again to every environment, it’s forced.

Chapter 8, Implement Infrastructure and Configuration as Code, will discuss IaC in more detail.

Test automation

To continuously deliver value to your end users, you must release fast and often. This has implications for the way you test your application. You can no longer execute manual tests when you release your application every few minutes. This means that you must automate as many of your tests as possible.

You will most likely want to create multiple test suites for the applications that you run at different stages of your delivery pipeline. Fast unit tests that run within a few minutes and that are executed whenever a new pull request is opened should give your team very quick feedback on the quality of their work, and it should catch most of the errors. Next, the team should run one or more slower test suites later in the pipeline to further increase your confidence in the quality of a version of your application.

All of this should limit the amount of manual testing to a bare minimum and allow you to automatically deploy new versions of your application with confidence.

Chapter 10, Integrating Continuous Testing, will cover test automation in detail.

Application performance monitoring

This last practice is all about learning how your application is doing in production. Gathering metrics such as response times and the number of requests will tell you about how the systems are performing. Capturing errors is also part of performance monitoring and allows you to start fixing problems without having to wait on your customers to contact you about them.

In addition to that, you can gather information about which parts of the application are more or less frequently used and whether new features are being picked up by users. Learning about usage patterns provides you with great insights into how customers use your applications and common scenarios they are going through.

Chapter 11, Managing Security and Compliance, and Chapter 12, Application Monitoring, will go into detail about both your application and your users’ behavior in production.

DevOps habits

The seven habits of successful DevOps teams are more concerned with culture and your attitude while developing and delivering software, and less with technical means than DevOps practices are. Still, it is important to know and understand these habits since they will help make DevOps adoption easier.

You will notice that developing these habits will reinforce the use of the practices enumerated previously and the tools you use to implement them. And of course, this holds the other way around as well.

Team autonomy and enterprise alignment

An important part of working Agile is creating teams that are largely self-directed and can make decisions without (too many) dependencies outside the team. Such a team will often include multiple roles, including a product owner that owns one or more features and is empowered to decide on their way forward.

However, this autonomy also comes with the responsibility to align the work of the team with the direction the whole product is taking. It is important to develop ways of aligning the work of tens or hundreds of teams with each other, in such a way that everyone can sail their course but the fleet as a whole stays together as well.

The best-case scenario is that teams take it upon themselves to align to the larger vision, instead of taking directions now and then.

Rigorous management of technical debt

Another habit is that of rigorous management of technical debt. The term debt in itself suggests that there is a cost (interest) associated with the delay in addressing an issue. To keep moving at a constant pace and not slowly lose speed over time, it is crucial to keep the number of bugs or architectural issues to a minimum and only tolerate so much. Within some teams, this is even formalized in agreements. For example, a team can agree that the number of unfixed bugs should never exceed the number of team members. This means that if a team has four members and a fifth bug is reported, no new work will be undertaken until at least one bug has been fixed.

Focusing on the flow of customer value

It is important to accept that users receive no value from code that has been written until they are using it. Focusing on the flow of value to a user means that code must be written, tested, and delivered and should be running in production before you are done. Focusing on this habit can drive cooperation between disciplines and teams.

Hypothesis-driven development

In many modern development methodologies, there is a product owner who is responsible for ordering all of the work in the backlog, based on the business value. This owner, as the expert, is responsible for maximizing the value that’s delivered by the development team by ordering all the items based on the business value (divided by effort).

However, recent research has shown that even though the product owner is an expert, they cannot correctly predict which features will bring the most value to users. Roughly one-third of the work from a team adds value for users, while another third decreases the value. For this reason, you can switch your backlog from features or user stories to the hypothesis you want to prove or disprove. You only create a minimal implementation or even just a hint of a feature in the product and then measure whether it is picked up by users. Only when this happens can you expand the implementation of the feature.

Evidence gathered in production

Performance measurements should be taken in your production environment, not (just) in an artificial load test environment. There is nothing wrong with executing load tests before going to production if they deliver value to you. However, the real performance effort is done in the production environment. It should be measured there and compared with previous measurements.

This also holds for usage statistics, patterns, and many, many other performance indicators. They can all be automatically gathered using production metrics.

Live-site culture

A live-site culture promotes the idea that anything that happens in the production environment takes precedence over anything else. Next, anything that threatens production, is about to go to production, or hinders going to production at any time gets priority. Only when these are all in order is the attention shifted to future work.

Also, a part of a live-site culture is ensuring that anything that disturbed the operation of the service is thoroughly analyzed – not to find out who to blame or fire but to find out how to prevent this from happening again. Prevention is preferably done by shifting left; for example, by detecting an indicator of a repeat incident earlier in the pipeline.

Managing infrastructure as a flexible resource

Finally, a successful DevOps team treats its servers and infrastructure as mutable assets that can be continually changed, upgraded, tuned, or even decommissioned to meet the demands of the business. The ability to do this is fueled by configuration and IaC. This may even go so far as creating a new production environment for every new deployment and just deleting the old production environment after switching all traffic from the old environment to the new one.

Besides keeping these DevOps practices and habits in mind, there are certain stages that you will go through while trying to move to a DevOps culture in your organization. The next section will take you through them.

 

The five stages of the DevOps evolution

When you are trying to move to a DevOps culture in your organization, it is going to take time. There are motions you have to go through while everyone in your organization embraces the changes they have to make to their ways of working. Others that have gone before you have gone through the following five steps or stages, which may help you. Knowing about them can help you accelerate your journey. These steps were first published in the 2018 State of DevOps Report and are discussed in the following sections.

Normalizing the technology stack

A common first step on the road to a DevOps culture is adopting. At a minimum, there are good tools for source control and often, a company standard and continuous integration and delivery are rolled out. Teams also work together to normalize the stack they develop software for. For example, one or two cloud vendors are chosen, and other deployment platforms are phased out. The same goes for tools for other purposes – they are standardized where possible. Homebrewed solutions are replaced with industry standards.

Standardizing and reducing variability

At this stage, teams work on further reducing the variation between and within applications and the development and operations teams that work on them, working together on aligning operating systems, libraries, and tools. Also, in this stage, deployment processes are changed to reduce the amount of variation between them, and configuration and infrastructure are often moved to source control.

Expanding DevOps practices

The remaining issues between development and operations are cleaned up, ensuring that the outputs of the development team are precisely what the operations team expects. Also, collaboration starts to grow between the two and they can work together without external dependencies on creating and delivering changes.

Automating infrastructure delivery

At this stage, the infrastructure that is used by developers and operations becomes fully aligned. Everything is deployed from source control and the same scripts or solutions are used by both teams.

Providing self-service capabilities

Before DevOps, virtual machines or hosting environments were often requested from operations, by developers manually, or through ticketing systems. Provisioning was done manually by operators, which could take days or sometimes even weeks.

Self-service capabilities mean that environments are no longer created manually, but through self-service APIs that operations teams make available to developers.

This way, developers can create and destroy environments on their own. They can create and test changes on their own and send them off or schedule them for automated deployment.

 

Summary

In this chapter, you learned what DevOps is (and what it is not) and its relationship with Agile. Moving to a DevOps culture helps you break down conflicting targets for developers on one side and operators on the other. This empowers them to work together on continuously delivering value to your end users, organizing their work in a single backlog, and working off a single board, while respecting the differences in their ways of working. Organizing developers and operators in product-oriented teams is the next important step in creating like-minded, goal-oriented teams.

Moving to DevOps can bring many benefits and you now know how these can be measured so that you can continuously keep improving. Next, you learned about the DevOps habits and practices that many successful DevOps teams exhibit. Mastering these yourself and with your team will enable you to go through a DevOps evaluation. All this helps continuously deliver value to your users.

In the next chapter, we will discuss Site Reliability Engineering (SRE) and how it complements DevOps for managing the reliability and scalability of your application.

 

Self-practice exercise

Make use of the concepts presented in this chapter to complete the following activities:

  1. Identify 2-3 DevOps-related metrics that may be important for your team.
  2. For each metric, identify the DevOps practices that apply.
  3. Define the current baseline for these metrics and list the areas of improvement.
 

Questions

As we conclude, here is a list of questions for you to test your knowledge regarding this chapter’s material. You can find the answers in the Assessments section:

  1. True or false: Development and operations departments often have conflicting goals.
  2. True or false: The seven DevOps practices discussed in this chapter are unrelated and one can be easily practiced without the other.
  3. Which of the following is not a part of the five stages of DevOps evolution?
    1. Normalizing the technology stack
    2. Automating infrastructure delivery
    3. Standardizing and reducing variability
    4. Hiring dedicated automation teams
  4. What is fastlaning?
  5. Describe in your own words, in a few lines, what the essence of DevOps is.
 

Further reading

There are many other resources that you may find helpful to learn more about DevOps culture and the DevOps way of thinking. Some of them are as follows:

About the Authors
  • Subhajit Chatterjee

    Subhajit Chatterjee has a Bachelor of Engineering and a graduate diploma on Information Technology. He has also taken up many online certifications that has helped him learn and grow as a software engineering professional. He has close to two decades of exposure to designing, implementing, and managing software development projects, using Microsoft & Open-Source technologies. He is a seasoned Engineering leader and has delivered many large and complex projects in Azure, IoT, Enterprise Integrations, Web Applications and Mobility space. He loves to solve problems and has had the privilege to work with customers across the globe. He is passionate about sharing his learnings with the community and enjoys coaching and mentoring his team members.

    Browse publications by this author
  • Swapneel Deshpande

    Swapneel Deshpande is a Solution Architect, Development Consultant, and a Trusted Technology Advisor, entrepreneur, start-up consultant and more. He is working in IT for nearly 2 decades with expertise in .NET and web-related technologies. He has led the software architecture design, development, and delivery of large, complex solutions. He is passionate about embracing new technologies and teaching. He is currently working with Microsoft and his current role involves leading large and complex projects right from architecture to delivery.

    Browse publications by this author
  • Henry Been

    Henry Been has been working in IT for over ten years. He is an independent architect, developer, and trainer in a number of companies. With many of these companies, he has embarked on a journey implementing practices such as continuous integration and deployment, infrastructure as code, trunk-based development, and implementing feedback loops. Alongside his work, he creates online training courses for A Cloud Guru, and frequently speaks at meetups and conferences. He was awarded the Microsoft MVP award in 2019.

    Browse publications by this author
  • Maik van der Gaag

    Maik van der Gaag is an architect and trainer at 3fifty, an experienced consultancy company with a strong focus on the Microsoft cloud. He has over 15 years' experience of providing architecture, development, training, and design expertise. During his career, he has worked on a variety of projects, ranging from cloud transformations to DevOps implementations. He loves to share his knowledge, which was also one of the reasons why he founded the Dutch Cloud meetup. Maik is a public speaker, writes blogs, and organizes events.

    Browse publications by this author