In the last chapter, we saw how infrastructure in the cloud works from not only a native but also a hybrid perspective. We will now take a look at how we can build resources in Azure. You can now go to the Azure portal and create any Azure resource; however, this can be very cumbersome without automation. Automation in Azure is accomplished through Azure DevOps and Azure Resource Manager (ARM) templates. We are strictly sticking to out-of-the-box Microsoft solutions, but there are quite a few other deployment and development tools available that can help you accomplish automation tasks. Once you've deployed your resources, you need to ensure they're secure.

In this chapter, we are going to cover the following:

• Azure DevOps and why it is important
• ARM templates and the different ways they can be used
• Fundamentals and best practices of deploying Azure Infrastructure as Code (IaC)
• Benefits and best practices for identity and access control in Azure
• Azure governance

Before we dive into how you can accomplish automation and what it means to have IaC, let's get an overview of Azure DevOps and ARM templates to build a foundation for this automation approach.

While this chapter isn't about Azure DevOps, it's a good idea to begin with a fundamental understanding of what it brings to the table. Azure DevOps is both a developer tool and a business tool, as it can be the source of truth for your code base and a backlog of items that code needs to accomplish. Let's look at some of the options that it brings to the table, from which you can pick and choose:

• Azure Repos allows you to either create a Git repository or Team Foundation Version Control to store your development source control.
• Azure Pipelines, one of the critical processes we will use in this chapter for the artifacts we create, provides build and release services for continuous integration and delivery (CI/CD) of your apps.
• Azure Boards helps deliver a product backlog to plan and track work, code defects, and other issues that may arise during your software development.
• Azure Test Plans allows you to test the code within your repository and enables you to perform manual and exploratory testing, along with continuous testing.
• Azure Artifacts provides the elements needed for your code to be packaged and deployed, such as NuGet resources usually shared with your CI/CD pipelines.

As you can see, Azure DevOps is Microsoft's tool for deploying and managing applications within Azure as part of the release management process. To learn more about Azure DevOps, you can head over to the documentation at https://docs.microsoft.com/azure/devops/user-guide/what-is-azure-devops?view=azure-devops.

Note

Azure DevOps is available for free with a five-user license, so feel free to have a look and explore how deployment in Azure works. Head to https://azure.microsoft.com/services/devops/ to create your free account.

Now that we've discussed some of the toolings at a very high level, let's take a look at ARM templates.

ARM templates

ARM templates are how your infrastructure is represented as code. ARM templates help teams take a more agile approach to deploying infrastructure in the cloud; it is no longer necessary to click deploy within the Azure portal to create your infrastructure. An ARM template is a mixture of a JSON file representing the configuration of your infrastructure and a PowerShell script to execute that template and create the infrastructure.

The real benefit of using the ARM template system is that it allows you to have declarative syntax. That means you can deploy a virtual machine and create the networking infrastructure that goes around it. Templates end up providing a process that can be run repeatedly in a very consistent manner. They manage the desired state of the infrastructure, meaning a template becomes the source of truth for those infrastructure resources. If you make changes to your infrastructure, you should do that through the templates.

The template deployment process can't be accomplished without orchestrating how the template process needs to run and what order it needs to run in. It is also useful to break these files into smaller chunks and allow them to be linked together or reused in different fashions with other templates. This can help with understanding and controlling your infrastructure while making it repeatable and stable. ARM templates are used in CI/CD pipelines and code deployment to build a suite of applications within the organization.

The following JSON file shows you how ARM templates are structured:

{
  "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
  "contentVersion": "1.0.0.0",
  "parameters": {},
  "variables": {},
  "resources": [],
  "outputs": {}
}

As you can see, there are several important parts: parameters, variables, resources, and outputs. Let's discuss each of them briefly:

• ARM template files should be parameterized, and there is a separate file for the parameters that maps to the parameters list in the JSON template file.
• The variables portion is for variables used within this file. Variables are generally used for creating naming functions to help generate a naming convention that is already structured, and for building that structure to use input parameters to make the name.
• The resources section is where all of the resources that you're trying to deploy with this ARM template are represented; these can range from virtual machines to websites.
• Finally, the outputs section is anything you want to pass out of your ARM template to be used elsewhere, such as an SQL Server name, before running your SQL scripts.

Three files are created when you create an ARM template file within Visual Studio:

• The first is the JSON file, which is the template that is represented in the preceding code.
• The second is the JSON parameter input file, a file that can be changed with every deployment to match the environment you want to deploy.
• The third is the PowerShell script used to execute the template. The PowerShell script accepts the resource group's inputs, the ARM template file, and the parameters file.

That was a quick overview of Azure DevOps and the files that are created when you create an ARM template. Let's see how Azure resources are deployed using these ARM templates.

As we saw in the ARM templates section, we want to use ARM templates to deploy our infrastructure, as it fits in nicely with our CI/CD process. There are several ways to approach building out these templates.

One way is to create a monolithic template that contains all of the resources that you want to deploy. To make things a little more modular, you could use a nested template structure. Alternatively, you may want to take a more decoupled approach and create smaller templates that you can link together, making a highly usable and repeatable structure.

Let's take a look at each of these methods, starting with the monolithic view:

Figure 2.1: Monolithic ARM template

As you can see in Figure 2.1, a monolithic ARM template deploys a UI front end with an API middle tier connected to the SQL database. In this process, we need to build out all of the dependencies within the JSON template. The SQL database is deployed before the API middle tier to use the connection string in the API application configuration. You would then deploy the UI layer with the API URL being used in the UI application configuration. The chaining of the deployment can work not only for deploying code but also helping with the configuration.

Alternatively, you could implement a nested template arrangement:

Figure 2.2: Nested ARM templates

As you can see, this is similar to the structure in Figure 2.1. However, the templates within this structure are nested in separate file sections. This means that each template owns the resource within it that it's trying to deploy. This structure is similar to breaking out your C# code into manageable methods and actions. This follows the same deployment process as discussed in the monolithic scenario, but the files are nested.

The final structure is linked ARM templates:

Figure 2.3: Linked ARM templates

As you can see, the templates are initially separate and decoupled from each other, and then we link them together in our release pipeline. Linked templates are similar to nested templates, except the files are external to the template and system, whereas the nested templates are included in the same scope as the parent template. This helps with reusability down the line, because the templates are separate files that can be linked to other deployment files.

We should note that with linked or nested templates, the deployment mode can only be set to Incremental. However, the main template can be deployed in Complete mode, so if the linked or nested templates target the same resource group, that combined deployment will be evaluated for a complete deployment; otherwise, it will deploy incrementally. To learn more about ARM deployment modes, visit https://docs.microsoft.com/azure/azure-resource-manager/templates/deployment-modes.

We've seen different ways of using these ARM templates to automate the deployment of infrastructure; now we turn to the benefits of doing so.

The main benefit of using IaC is automating the creation, updating, and configuration of different resources and environments. Automation takes the human element out of the picture while adding certain key benefits:

• The ability to schedule automated deployments, helping your operations staff work fewer hours.
• The ability to smoke test your automated deployments.
• The ability to create a repeatable process.
• Pure self-healing applications can be achieved.
• The ability to roll back changes.
• Resource Manager helps in tagging resources.
• ARM takes care of the dependencies of resources in the resource group.

Automation at any point is the real key to using a platform like Azure, whether that is automating your infrastructure deployments or testing to ensure the stability of your production deployments.

In order to reap these benefits, we need to learn how to use ARM templates effectively.

We want to take a quick look at some best practices to optimize ARM templates. But first, let's start by understanding what some of our template limits are.

Overall, the template can only be a maximum of 4 MB, and each parameter file is limited to 64 KB. You can only have 256 parameters, with 256 variables, containing 800 resources, 64 output values, and 24,576 characters in a template expression. As we've discussed, you can exceed some of these limits by using nested templates if your template gets too big, but Microsoft recommends that you use linked templates to help avoid these limits. In the following sections, we discuss some best practices for each component within an ARM template.

Parameters

The ARM template system within Azure DevOps resolves parameter values before deployment operations and allows you to reuse the template for different environments. It is essential to point out that each parameter must have a set data type value. You can find a list of these data types at https://docs.microsoft.com/azure/azure-resource-manager/templates/template-syntax#data-types.

Best practices

Microsoft recommends the following best practices for parameters:

• It's best to minimize the use of parameters. As we pointed out at the beginning of the chapter, you should use variables for properties and only use parameters for the things you need to input.
• It is recommended that you use camel casing for parameter names.
• It is also recommended that you describe each parameter, so when other developers use the template they know what the parameters are.
• Ensure that you use parameters for those settings that may change when the environment changes, such as capacity or app service names.
• Ensure you name your parameters to make them easily identifiable.
• Provide default values for parameters; this involves providing the smallest virtual machine skew size so non-production environments use smaller resources and other developers that use the template have a basic starting point.
• If you need to specify optional parameters, avoid using empty strings as the default value and instead use a literal value. This helps to provide a naming structure for users of the template.
• Try to use allowed values as little as possible, as these may change over time and can become difficult to update in your scripts.
• Always use parameters for usernames and passwords or secrets to be set for each environment and not hardcoded in the template. You should also use a secure string for all passwords and secrets.
• When you need to set a location for the resource you're deploying, set the default value to resourcegroup().location so the location value is set correctly within the resource group.

As you can see, parameters are very useful in the ARM template process because they allow us to be flexible with the environments we're trying to deploy. Remember to keep these templates as simple as possible with the applications or microservices you're trying to deploy.

Variables

Variables are also resolved before starting the deployment, and the resource manager replaces the variable with its determined value. Variables are useful in deriving complex naming within your template and allow you to only pass in the required parameters.

An example of this is an organization that uses a customer ID and depends on this for its naming convention to keep all deployed resources in Azure unique to that customer ID. In this case, you would create the customer ID as a parameter and then develop variables to generate names using your naming standard. You can find a list of acceptable data types for variables at https://docs.microsoft.com/azure/azure-resource-manager/templates/template-syntax#data-types.

Best practices

Microsoft recommends the following best practices for variables:

• Remember to remove unused variables and files as they can be confusing.
• Use camel casing for your variable names.
• Use variables for values that you need more than once within your template.
• Variable names must be unique.
• For repeatable patterns of JSON objects, use the copy loop in variables.

Resources

The resources section of the ARM templates is reserved for resources that will be deployed or updated. ARM templates generally help derive the desired state of the resources within Azure. When changing Azure infrastructure, it is always a good practice to change your template first and then re-run it to change your Azure resources. All too often, organizations make changes on the portal but forget to change their ARM template, and then the next time they deploy these resources, they are deployed into the wrong state.

Best practices

Microsoft recommends the following best practices for resources:

• Add comments to your resources so that others know their purpose.
• Remember that there are quite a few resources that require unique names, so never hardcode your resource names.
• When you add a password to a custom script extension, use the CommandToExecute property in the protected settings of Azure Resource Manager.

We now have a fundamental understanding of the elements within an ARM template. Our next focus will be identity and access control once your resources have been deployed.

Before we dive in, it is good to realize that there is a shared responsibility between you and the cloud provider when it comes to security and securing your resources. It is essential to understand where your responsibility stops and the cloud provider steps in. Let's take a quick look at shared responsibility in Azure, as you can see in Figure 2.4:

Figure 2.4: Azure shared responsibility

Depending on the type of service you choose in Azure, your responsibilities will vary, as will those of the cloud provider. Note the differences between on-premises resources and the various Azure options. You can see that, regardless of any new responsibilities, you will always retain the responsibility for endpoints, account management, accounts, and data repositories you create in the cloud.

With Azure being a public-facing resource, security is at the forefront of its development. There is a wide range of tools and advisors within Azure that help you take advantage of Azure's different security tools and capabilities.

What are the security benefits of Azure?

Organizations' on-premises security groups only have limited resources (team members and tools) to view exploits and attackers. One of the benefits of using a platform like Azure is that you can offload those responsibilities to the provider in the cloud and gain a more efficient and intelligent approach to your organization's threat plane without the need to bring in physical resources.

One of the most significant benefits of Azure is that when you create a tenant, it comes with Azure AD behind it, which allows you to start from a security perspective in Azure. Azure AD is used to lock down all of the services and resources within Azure. You can also use Azure AD to secure your applications or create Business to Customer (B2C) or Business to Business (B2B) ADs to house your client information.

Helpful tools in Azure

There are several security and audit solutions built into Azure to strengthen your security posture, which can be viewed in the Azure portal through the security and audit dashboard from your home screen. Here are some helpful tools in Azure to assist you and your organization:

• We mentioned earlier that Azure Resource Manager helps keep everything in one place for deploying, updating, and deleting resources within your solution to support coordinated operations.
• Azure also offers Application Performance Management (APM), which is referred to as Application Insights. Application Insights gives you the ability to monitor your applications within Azure and detect performance anomalies.
• Azure Monitor allows you to visualize your infrastructure for your activity log and the individual diagnostic logs for your Azure resources.
• Azure Advisor is like a personalized cloud concierge to help you optimize your cloud resources. This service can help detect security and performance issues within your applications.
• Azure Security Center helps prevent, detect, and respond to different threat planes for your applications within Azure. It helps provide security monitoring and policy management across all of your other subscriptions.

Best practices

Here are some Azure security best practices recommended by Microsoft:

• Use Azure AD for central security control and identity management in Azure. This will make management and integration more streamlined.
• Try to keep your Azure AD instances to a single source of truth.
• If you have an on-premises AD, it is recommended that you integrate it with Azure AD, using Azure AD Connect for a single sign-on experience.
• If you use Azure AD Connect to sync your on-premises AD with Azure, turn on password hash synchronization in case the main resource goes offline or is deprecated.
• Remember that you can use Azure AD for authentication in your new applications, and this can be accomplished through Azure AD directly, B2B, or B2C.
• Use management groups to control your access to subscriptions. This helps with centralized management, over needing to worry about Azure Identity and Access Management (IAM) in each subscription.
• Use Conditional Access for your support personnel so that they can elevate their permissions when needed in Azure, rather than having access all the time.
• Block legacy protocols that aren't used to stop attack planes.
• It is recommended you use self-service password resets for your users if you're using Azure AD for your applications and you want to ensure you monitor this process.
• If you are using Azure AD Connect, ensure that your cloud policies match your on-premises policies.
• Enable multi-factor authentication for your organization if possible.
• If you wish to provide built-in roles in Azure, ensure that you maintain role-based access over rule-based access, as rule-based access can be very cumbersome to manage in the long run.
• Ensure that you give the least privileged access to those logging in to Azure, so that when an account is compromised, its access is limited.
• Define at least two emergency access accounts, just in case the organization admin operation team members' accounts get compromised.
• Ensure you control locations where resources are created for your organization if you also wish to monitor suspicious activities within your Azure AD tenant actively, as there may be data restrictions in different regions.
• Ensure you use authentication and authorization for your storage accounts.
• Review and apply improvements to your best practices over time.

Now that we have an understanding of security in Azure, we can review how to use Azure governance.

Azure governance is defined as the processes and mechanisms that are used to maintain control of your applications and resources in Azure. They are the strategic priorities involved in planning your initiatives within your organization. Azure governance is provided in two services: Azure Policy and, as we will discuss in Chapter 7, Offers, support, resources, and tips to optimize cost in Azure, Azure Cost Management.

The purpose of Azure Policy is to organize your operational standards and to assess your compliance. Azure Policy regulates compliance, security, costs, and management while implementing governance for consistency of your resources. Everything that we see in Azure is governed by these policies, which contain business rules in JSON format and policy definitions. The schema for these policy definitions can be found at https://schema.management.azure.com/schemas/2019-09-01/policyDefinition.json.

What are the benefits?

Azure governance and Azure Policy help with building and scaling your applications while maintaining a level of control. This helps create guardrails and assists with deploying entirely governed environments throughout your organization's subscription using Azure Blueprints. As we will discuss in Chapter 7, Offers, support, resources, and tips to optimize cost in Azure, it also assists in managing costs by providing insights into your spending to maximize your cloud investment. In addition, Azure governance offers the following benefits:

• Helps with audit and enforcement of your policies for any Azure service
• Helps encourage accountability throughout the organization while monitoring spending
• Creates compliant environments, including resources, policies, and access control
• Helps ensure compliance with external regulations via built-in compliance controls

In the following sections, we'll look in detail at some of the features and services available via Azure governance, namely Azure management groups, Azure Policy, Azure Blueprints, Azure Graph, and Azure Cost Management and Billing.

Azure management groups

Azure management groups help manage your Azure subscriptions by grouping them and taking actions against those groups. They allow you to define security, policies, and typical deployments via blueprints. They help create a hierarchical view of your organization so that you can efficiently manage your subscriptions and resources:

Figure 2.5: Azure management groups

As you can see in Figure 2.5, there is a simple separation between production and non-production. We should understand from this illustration that each management group has a root or parent that every hierarchical structure inherits from. You can create a global structure at the root, or you can create a production or non-production policy in either one of the branches.

Azure Policy

Azure Policy was put in place to help enforce asset compliance in organizational standards within Azure. Common uses of Azure Policy are implementing governance for resources consistency, security, costs, and management. Like everything in Azure, Azure policies are in JSON format, and you add business rules for policy definitions to help simplify the management of these rules.

Azure policies can be applied to Azure resources in different life cycles or during an ongoing compliance evaluation. These can be used as a control mechanism to deny changes or to log them. The difference between Azure Policy and Azure rule-based access control (RBAC) is that your policy does not restrict Azure actions. This means a combination of Azure RBAC and Azure policy provides the full scope of security in Azure.

The following best practices are recommended by Microsoft:

• When you use Azure policies, it's always good to start with an audit policy rather than a denial policy, as setting a denial policy may hinder automation tasks when creating the resources.
• When creating your definitions, consider your organizational hierarchies. Creating higher-level definitions such as at the management group or subscription level is recommended.
• Create and assign initiative definitions or policy sets even for the smallest policy definitions.
• It is good to remember that once an initial assignment is evaluated, all policies within that initiative are evaluated as well.
• You should think about using policies to help control your infrastructure, like requiring antivirus to be installed on all virtual machines or not allowing specific sizes of virtual machines to be created in a non-production environment. To gain a better understanding of Azure policy definition structure, you can learn more at https://docs.microsoft.com/azure/governance/policy/concepts/definition-structure.

Azure Blueprints

Azure Blueprints enables technology groups to develop a repeatable set of Azure resources that support an organization's patterns, requirements, and standards. Blueprints are a great way to orchestrate the deployment of various resources, such as role assignments, policies, ARM templates, and resource groups. Azure Blueprints is an extension of ARM templates, which are designed to help with environment setup, and Azure Blueprints uses templates to accomplish this goal.

Azure Resource Graph

Azure Resource Graph was created to extend ARM templates' capabilities to help explore resources even across subscriptions. Azure Resource Graph queries allow you to search for complex results from resources that have been deployed in Azure. Azure Resource Graph is the query system that supports the search in Azure. The query language is based on the Kusto Query Language, which is also used by Azure Data Explorer, so it may be new to you and take a little bit of getting used to.

You need the appropriate rights in Azure RBAC to see the resources—this is the read permission. If you don't get any results returned in Azure when you use Azure Resource Graph, check your read permission first.

Azure Resource Graph is free to use, but it's throttled to ensure the best experience for everyone.

Azure Cost Management and Billing

Azure Cost Management and Billing was created to help analyze, manage, and optimize the costs of your workloads in Azure. It was introduced to help businesses reduce their risk of potential waste and inefficiencies as they migrate to the cloud. Azure Cost Management and Billing does the following:

• Assists in paying your bills
• Generates monthly invoices containing cost and usage data that can be downloaded
• Sets spending thresholds
• Analyzes your costs proactively
• Identifies opportunities to optimize spending for your workloads in Azure

We will cover this in more depth in Chapter 7, Offers, support, resources, and tips to optimize cost in Azure, since individuals and organizations manage cost streams differently in Azure.

As you can see from this chapter, Azure and Azure DevOps have significant synergies. It is essential to create a repeatable, stable way to deploy your code and infrastructure to the cloud. While learning how to deploy this infrastructure to the cloud, we needed to understand why ARM templates are used. This led us to discuss some of the fundamentals and best practices around deploying this infrastructure to leverage our code or applications. We looked at exactly how we need to secure resources and our applications through identity and access control. This brought us to understanding how to create governance in Azure to ensure consistency and compliance.

Azure creates a lot of frameworks that allow you to digest the things that you need to leverage to ensure your applications are secure and complete. It is also good to remember that as you approach Azure, you should approach it from an automation perspective. An organization should develop compliance, Azure governance, and best practices that work across the organization while minimizing its business risk.

Now that we have built a foundation, in the next chapter, we will move on to how we can modernize applications.

• Azure Cost Management and Billing: https://docs.microsoft.com/azure/cost-management-billing/cost-management-billing-overview
• Azure Resource Graph: https://docs.microsoft.com/azure/governance/resource-graph/
• Azure Blueprints: https://docs.microsoft.com/azure/governance/blueprints/overview
• Azure Policy: https://docs.microsoft.com/azure/governance/policy/overview#policy-definition
• Azure Policy: https://docs.microsoft.com/azure/governance/policy/concepts/definition-structure
• Azure management groups: https://docs.microsoft.com/azure/governance/management-groups/
• ARM Template Toolkit: https://docs.microsoft.com/azure/azure-resource-manager/templates/test-toolkit
• ARM Template Structure: https://docs.microsoft.com/azure/azure-resource-manager/templates/template-syntax
• ARM Template Recommendations: https://docs.microsoft.com/azure/azure-resource-manager/templates/templates-cloud-consistency
• Azure Security Center: https://azure.microsoft.com/services/security-center/
• Azure Security Best Practices: https://azure.microsoft.com/blog/azure-storage-support-for-azure-ad-based-access-control-now-generally-available/
• Resources Link: https://docs.microsoft.com/azure/azure-resource-manager/templates/template-syntax#resources
• ARM Template Best Practices: https://docs.microsoft.com/azure/azure-resource-manager/templates/template-best-practices?WT.mc_id=azuredevops-azuredevops-jagord
• Variables: https://docs.microsoft.com/azure/azure-resource-manager/templates/template-variables
• Parameters: https://docs.microsoft.com/azure/azure-resource-manager/templates/template-parameters