This chapter is an introduction to DevOps architecture for the enterprise. First, we'll look at the business of an enterprise. The business sets its goals and with that, defines the criteria for IT delivery, which supports these business goals. Therefore, the DevOps architecture must be aligned with the enterprise architecture. In this chapter, we will learn how to set up the reference architecture and design the different DevOps components while working with the VOICE model. Next, we'll learn how to deal with service levels and key performance indicators in DevOps models.
By the end of this chapter, you will have a clear view of how to start using the architecture and defining a DevOps strategy. An important lesson you'll learn in this chapter is that setting up DevOps in an enterprise becomes more complicated when organizations have outsourced large parts of their IT delivery. During this chapter, you will learn how to engage DevOps in enterprises with sourcing models.
We're going to cover the following main topics:
This book will focus on implementing and scaling DevOps in large enterprises. Before we get into the specific challenges of an enterprise, we need to have a common understanding of DevOps.
Somewhere, businesses and their leaders must have thought that it was a good idea to put developers and operators into one team. In essence, DevOps is the development and operations stages working as one team, on the same product and managing it. You build it, you run it.
DevOps has gained a lot of momentum over the past decade, especially in enterprises. But implementing DevOps turned out to be quite difficult. The reason for this is that enterprises are not organized in a structure that works for DevOps. From the last century onward, most enterprises outsourced a lot of their IT. Most of the IT muscles of a major enterprise are therefore still with system integrators and software houses. DevOps becomes more difficult when development is done by a software house and operations is outsourced to a system integrator.
DevOps starts with the business. By bringing teams together into a development and operations environment that traditionally work in silos, an enterprise can speed up development and release new products and services. The rationale behind this is that less time is needed to do handovers between development and operations. Also, by removing the barrier between development and operations, the quality of products will improve since DevOps includes quality assurance, testing, and security. Customer feedback is continuously evaluated and included in new iterations of the product.
The benefits of DevOps are as follows:
It starts with the business and thus the starting point is the enterprise architecture. This is where the business goals are set and we define how these goals will be met. IT delivery is key to meeting these goals. In large enterprises, the architecture also defines the IT delivery processes and the demarcation between these processes. We will look at IT delivery and its processes in more detail in the next section.
As we mentioned at the beginning of this section, large enterprises typically have an operating model that is based on outsourcing. This makes implementing DevOps more complicated. The enterprise architect will have to have a very clear view of the demarcation between the different processes and who's responsible for fulfilling these processes. Who is responsible for what, when, and why? The next question is, how does it map to DevOps?
First, we need to understand what the main processes are in IT delivery. These processes are as follows:
One important remark that must be made is that when we refer to a product, we are talking about a product that is code-based.
Tip
There's one major movement in IT delivery: everything in IT is shifting to code. It's one of the main principles of The Modern DevOps Manifesto: Everything is code. It applies to applications, but also to infrastructure components such as network devices, servers, and storage devices. Therefore, DevOps not only includes software development and operations for applications, but also for infrastructure with Infrastructure as Code and Configuration as Code. Public clouds such as AWS, Azure, and Google Cloud Platform play a significant role in these developments.
In other words, the team is developing code: application code, Infrastructure as Code, and also test code. A developer will work on a specific piece of code that has been defined in the product backlog. The whole end product – for instance, an application – has been broken down into product backlog items (PBIs), where each developer will work on a PBI. As soon as a piece of code is ready, it needs to be tested on itself, but also as a component of the end product. Due to this, in development, code needs to be merged. This merging process is triggered by a pull request, where the developer requests to have the code merged and joined to the end product, thus fulfilling the user story. This is done using pipelines.
In Chapter 2, Managing DevOps from Architecture, we will discuss setting up and managing pipelines, both for application development and for infrastructure.
We can divide the whole DevOps cycle into two major phases called deployment and operations, as follows:
a) Request fulfillment
b) Incident management
c) Problem management (postmortem)
d) Configuration management
e) Change management
But DevOps adds something to this; that is, continuous integration and continuous delivery (CI/CD):
CI/CD requires a feedback loop to make it continuous. It needs feedback about the delivered products and services. This is then looped back to the developers and from there, new iterations are planned to improve the product or service.
This works well if an enterprise controls the full cycle, but large enterprises have outsourced a vast number of activities to other companies. The rationale behind this sourcing strategy is typically because a certain activity is not perceived as a core activity, and it can be done more cost-effectively by a company that specializes in such activities.
However, enterprises have gone through a massive change over the last decade. IT has become more and more important and, in some cases, has become a core activity. Banks are a good example. Banks are IT companies nowadays, and the output of their IT delivery is financial products. Due to customer demands, releases of these products with new features have become more frequent, with up to several releases per day. The consequence of this is a major shift in IT delivery itself.
The next few sections will discuss how IT delivery works in sourcing models and how it impacts successfully implementing DevOps.
In this section, we will look at the sourcing model in large enterprises. This can be quite complicated, but if we learn to think in terms of sourcing tiers, it becomes more tangible and comprehensible. This is the target enterprise model, as shown in the following diagram:
Using this model, we can break down IT delivery into three tiers:
In practice, we see service providers also acting on tier 2; for instance, if they are involved in larger programs spanning multiple products. Tier 2 then becomes the orchestration level, where a provider is responsible for aligning different streams in the lower tier. The key takeaway is that tier 1 should always be the enterprise level, where the overall governance and architecture is defined.
In this section, we learned that a lot of enterprises have outsourced larger parts of their IT and that this can complicate the process of implementing DevOps. We learned that the strategy for the entire enterprise is at tier 1, the highest tier. DevOps sits on the lowest tiers, where projects are actually executed. This is the tier where the enterprise interfaces with sourcing companies. However, this only works if we have a clear view of the architecture. We will discuss this in the next section.
In the previous sections, we looked at the different processes in IT delivery, how it is integrated with DevOps, and how this is executed in sourcing models. We have learned that it starts with a clear architecture that clearly defines the processes.
Any DevOps architecture will have to address planning, development, integration, deployment, and operations. But we have to keep in mind why we are doing DevOps, which is to achieve business goals in a faster, more agile way where we continuously improve products. The DevOps architecture does not stand on its own; it has to be linked to the enterprise architecture.
An enterprise architect will most likely start from The Open Group Architecture Framework (TOGAF). TOGAF is globally accepted as the standard for enterprise and business architecture. It uses the Architecture Development Method (ADM) to draft the architecture. The ADM is shown in the following diagram:
Just like DevOps, the ADM is a cycle – except for the preliminary phase, which is where the need for an architecture is formulated. This has to be done at the tier 1 level in the sourcing model that we discussed in the previous section, IT delivery in sourcing models. The strategy and enterprise architecture is always set at tier 1.
The core of ADM is the sequence in architecture design, which is business first, and setting the principles and requirements for the actual solutions. These principles drive the architecture for data usage, the applications, and finally the technology that will be used. This is important because architecture is not about technology in the first place. Technology is purely the ability to achieve business goals at the enterprise level.
ADM assumes that the architecture is not static. It changes as soon as the business requirements change. If the business demands change, there will likely be a need to adapt the architecture and the forthcoming solutions. This is where TOGAF and DevOps meet, since the two frameworks complement each other.
The following table shows where the enterprise architecture and DevOps are complementary. To put it very simply, the enterprise architecture sets the strategic business goals, where DevOps translates this at a more tangible, tactical level and really tells us how to achieve these goals by developing products and executing operations. The following table shows the main differences between the enterprise architecture (EA) and DevOps:
In the next section, we will study the DevOps principles.
The enterprise architecture is executed on tier 1, the strategic level. This is where the goals are set for the entire enterprise. The next level is tier 2, where DevOps teams will translate the goals into product features and start developing. DevOps teams will have to work according to a set of principles.
In this section, we will look at the main principles for DevOps. In this book, we will use the six principles from the DevOps Agile Skills Association (DASA):
Adhering to these principles will lead to the following architecture statements, which are at the core of DevOps:
In this section, we looked at the IT delivery processes and how they impact DevOps. We learned that IT delivery is driven by business demand and that this business demand is the starting point for any architecture. This is included in the TOGAF framework for the enterprise architecture. After that, we mapped the enterprise architecture to DevOps principles.
In the next section, we will merge the DevOps principles for the architecture and the IT delivery principles into a reusable reference model.
The final step is to merge the DevOps principles into one model for our reference architecture. The model contains two circles. The outer circle is the product circle, while the inner circle represents the operational activities. As a logical consequence, the outer circle is governed by the enterprise itself.
The inner circle is about actually delivering the products using DevOps. There are interfaces between the outer and inner circle: collaboration, automation, integration, and configuration management.
The reference model is shown in the following diagram:
In the outer circle, the business goals are translated into the architecture. From the architecture, a portfolio is created with building blocks to create products and services. Products are released to the market and adopted, but due to changing demands, there will be requests for changes. These changes will drive enterprise planning and ultimately change the business goals, meaning that the business will constantly have to adapt to changing demands. This is the field of enterprise architecture.
The plans and the actual builds are executed in the inner circle. In this circle, the product is broken down into product backlog items that will be developed and eventually operated on by DevOps teams. These teams do not operate by themselves, but on triggers from the outer circle. That's what the interface layer is about – it's the interface between the business and the execution teams doing IT delivery. There's collaboration between architecture and development. Releases should be automated as much as possible, requests and changes must be integrated with planning and the backlog of the DevOps teams, and builds that are pushed to production must be monitored and brought under the control of configuration management so that the architecture and portfolio stay consistent in case of changes.
Let's have a look at how this would work in practice by plotting personas into the model. This will result in a DevOps workflow for enterprises, as shown in the following diagram:
Here, we have created a model where the enterprise has full control over its portfolio and products. Yet, it can improve quality and speed up delivery by working with combined, multidisciplinary teams – even those that come from different suppliers.
In the next section, we will study the final, lowest tier in our model and discover various DevOps components.
So far, we've learned how to start defining the architecture, looked at the architecture principles for DevOps, and drafted a reference architecture model. The next step is to look at the different components within DevOps. In this section, we will learn what components must be included in a DevOps architecture. This is tier 3 of our target enterprise model – the level where all the activities are executed.
The following diagram shows all the components that will be discussed briefly:
The reason that this has been presented as an infinite loop – or a pretzel – is because feedback from the live product that is managed by ops (operations) will be continuously looped back to dev (development) to improve the product.
The different components are as follows:
At this level, interoperability is crucial. Remember that large enterprises will likely work with several service providers, fulfilling parts of the IT delivery process. When we want all these companies to work together in a DevOps way, we need to make sure that the processes and tools are aligned. Next, we need to have a common understanding of the various activities that are executed as part of these processes. The key term here is consistency. All DevOps components must be defined and implemented in a consistent way. Every developer and operator must work according to the same definition and with the same components.
The main question is, in what stage should ops already be involved? The answer is, at the earliest stage possible, so indeed in the plan phase. Ops plays a key role in defining how products can be managed once they've gone live. They should set requirements and an acceptance criterion before going live. If a developer builds something that can't be managed by ops, the product will fail, and business demands will not be met.
The following table breaks down the components into activities:
In Chapter 2, Managing DevOps from Architecture, and Chapter 3, Architecting for DevOps Quality, we will dive deeper into this and how architects can improve their designs for these components using CI/CD pipelines to enable automation, collaboration, and integration.
In the next section, we will discuss the drivers for architecture from a business perspective, as laid down in SLAs and KPIs.
In the Understanding IT delivery in enterprises section, we learned that in DevOps, IT delivery and IT service management processes are still valid. Typically, enterprises contract SLAs and KPIs to fulfill these processes so that these enable the business goals. If one of the processes fails, the delivery of the product will be impacted and as an ultimate consequence, the business will not achieve its goals, such as an agreed delivery date or go live release of the product. Hence, understanding SLAs and KPIs is important for any architect. This is why it is included in the sourcing model that we discussed in the IT delivery in sourcing models section.
Service-level agreements are positioned between the tactical processes of DevOps and the strategic level at the enterprise level where the goals are set. SLAs and KPIs should support these goals and guide the DevOps process.
The six most important metrics that should be included in SLAs for DevOps are as follows:
This list is by no means exhaustive. Enterprises can think of a lot of different metrics and KPIs. But the advice here is to keep things simple. Keep in mind that every metric that is included in any contract needs to be monitored and reported, which can become very cumbersome. One more thing to remember is that the most important metric sits at the business level. Ultimately, the only thing that really counts is how satisfied the customer of the business is or, better said: what's the value that's delivered to the end customer?
In the final section of this chapter, we will elaborate on the term value by explaining the VOICE model.
DevOps teams need to deliver value to the end customer. The VOICE model, as defined by the IT company Sogeti, addresses this. VOICE stands for Value, Objectives, Indicators, Confidence, and Experience. The idea behind this model is that any IT delivery should deliver value to someone – typically, the end customer of a business. Value sets the objectives for IT delivery and these objectives are measured using indicators. Indicators also measure whether the pursued value will be achieved.
Confidence is about the indicators and if they contain relevant information to confirm that IT delivery actually results in the targeted value. Lastly, experience tells us if the delivered system is fulfilling the business demands and which improvements will lead to more business value. With that, the cycle starts over again.
This model is shown in the following diagram:
Since VOICE also involves looping feedback back to the beginning of the cycle with the aim of improving products and adding more value to the business, the model can be used for DevOps projects.
In Chapter 3, Architecting for DevOps Quality, we will explore VOICE in more detail.
This chapter was the introduction to DevOps for architects. We learned that the enterprise architecture sets the architecture principles for the entire enterprise by using the TOGAF methodology. The business goals are defined at the enterprise level. DevOps projects and teams are concerned with IT delivery and fulfilling the business' demands by building, deploying, and running IT systems.
DevOps needs to adhere to the business goals and, therefore, with the enterprise architecture. Yet, DevOps features a specific architecture that enables CI/CD in IT systems. Because of this, we learned about the six DevOps principles and how these are applied to a reference model in which the enterprise still has full control of the products, but multidisciplinary teams can work autonomously on them.
Next, we looked at the different DevOps components and KPIs to measure the outcomes of DevOps projects. The key takeaway from this is that every project needs to add to a better user experience and thus add business value. Due to this, we briefly studied the VOICE model.
In the next chapter, we will learn more about automation, collaboration, and integration by designing CI/CD pipelines.
Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.
If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.
Please Note: Packt eBooks are non-returnable and non-refundable.
Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:
If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:
Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.
You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.
Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.
When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.
For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.