This chapter introduces Microsoft Azure Storage, its types, and the differences between Azure's different models. It also introduces Azure Storage accounts and how to work with them. Moreover, you will have learned how to automate all of these tasks by the end of the chapter.

The following topics will be covered:

• An introduction to Microsoft Azure Storage
• Why Azure Storage?
• Azure terminologies
• Azure Service Management (ASM) versus the Azure Resource Manager (ARM) model
• Azure Storage types
• Azure Storage accounts
• Automating Azure tasks

Storage has always been one of the most important cornerstones of every system. You cannot imagine a virtual machine (VM), web application, or mobile application running without any sort of dependency on storage, and that is what we will cover throughout this book, but from the perspective of the cloud generally, and Azure specifically.

Microsoft Azure Storage is the bedrock of Microsoft's core storage solution offering in Azure. No matter what solution you are building for the cloud, you'll find a compelling use for Azure Storage.

Microsoft Azure Storage is not just a traditional storage system; it's scalable and can store up to hundreds of terabytes of data, meaning that it fits almost every scenario you can ever imagine in many fields, such as IT, science, medical fields, and so on.

At the time of writing, Microsoft Azure is generally available in 36 regions, with plans announced for six additional regions, as shown in the following figure:

This global presence means you can host your storage in the nearest region and access it from anywhere in the world. Considering that Microsoft continues to build new data centers in new regions, the latency between you and your services in Azure will decrease.

There are many reasons for using Azure Storage which will be covered throughout this book. Below is a sneak peak of a couple of them:

• Global presence: You can host your storage wherever you want in the available Azure regions, allowing you to provide applications close to your user base.
• Redundancy and recovery: As mentioned earlier in this chapter, Azure has a global presence which can be leveraged to maintain storage availability using data replication even if a disaster occurs in a region, which will be covered later in this chapter.
• Many flavors: Azure Storage has many flavors, based on resiliency, durability, connectivity, performance, and so on, which can be used according to your needs in different scenarios. This will be covered later in this chapter.
• Pay as you go: Pay as you go has always been one of the distinguished reasons for using the cloud generally. It is no surprise that Azure Storage supports this model as well.

Due to an overlap of terms and some misperceptions about the ways that Azure Services are delivered, terminology is a sticking point even for people who have been working with the technology for some time. The following table provides accurate but short definitions for terms related to Azure services. These definitions will be expanded upon in detail throughout the book, so don't worry if they are confused at first:

At the time of writing, Azure services are being provided via two portals, which follow two different models. The Azure classic portal follows the ASM model and the Azure portal follows the ARM model.

Historically, Azure services were provided via one portal prior to 2014, the classic portal, which can be accessed via the following URL https://manage.windowsazure.com/.

The model that was used for that portal is called the ASM model, within which each resource existed independently. You could not manage your resources together; you had to build up and track each resource. For example, you would have to manage storage from the storage blade, and the same goes for the virtual networks, VMs, and so on. So, when your environment got bigger, there would be chaos in the management scheme. You would have to know which VMs were stored in which storage account, and that could lead to some critical situations, such as reaching the IOPs limits of the storage account. In turn, this could result in you accidentally creating a new VM and assigning it to that storage account. As a result, the VM would run with terrible performance. This would not be your only concern when working with the ASM model; you might want to delete a solution with multiple resources, which you would have to do manually for each resource.

When you open the Azure classic portal, it will look like the following screenshot:

In 2014, Microsoft launched a new portal which follows a new model, called the ARM model. This portal can be accessed via the following URL https://portal.azure.com/.

This model depends on the concept of resource groups, which means you can group all your resources within a container, resulting in resources being deployed in parallel. As a result, you will not face the same problems as you did with the classic portal.

The following diagram describes the deployed resources through the ARM model:

Here are the benefits you will gain using this portal:

• Ability to manage your resources as a group instead of managing them separately.
• Use Role Based Access Control (RBAC) to control access to resources, so that you can assign permissions to a user on a resource or some resources but not to other resources (as it was in the classic portal).
• Use tags to organize and classify your resources, which can help you with billing. For example, you might want to monitor the billing of some resources that make up a solution, for example, a web server. By assigning a tag to the resources that make up that solution, you would be able to monitor its billing, and so on.
• Support the usability of JSON to deploy resources instead of using the portal.
  • Deploy resources in parallel instead of deploying them sequentially and waiting until every resource deployment finishes to deploy another one.
  • Specify dependencies during the deployment of resources. For example, a VM will not be created until a storage account and a virtual network are deployed because the VM VHD would need a place to be stored in and an IP Address from a virtual network./li>
  • Reuse the JSON template to deploy a solution with the same specifications.

• Resources with the same life cycle should be gathered in the same resource group.
• Resources in different regions can be in the same resource group.
• The resource cannot exist in multiple resource groups.
• A resource group supports RBAC, wherein a user can have access to some specific resources, but no access to others.
• Some resources can be shared across resource groups, such as storage accounts.
• ARM VMs can only be placed in ARM storage accounts.

When you open the Azure portal, it will look like the following screenshot:

Here are some things you need to consider:

• You cannot create a VM using the ARM model and assign it to a virtual network built using the ASM model
• You cannot use a prebuilt image that was created using ASM APIs to build a VM using the ARM model, but as a workaround, you can copy the VHD files from the storage account in the classic portal to a storage account created in the ARM model
• You can migrate assets from the ASM model to the ARM model
• Every resource must be assigned to a resource group, so whenever you want to move a resource between resource groups you must remove it from its current resource group, then add it to the new resource group.

Azure Storage has many types and even subtypes of those types in order to satisfy Azure services' consumer needs and to fit most scenarios.

The most common types can be classified based on the following factors:

• Durability (replication)
• Performance (Standard versus Premium)
• Persistency (persistent versus non-persistent)

One of the most buzzing questions about the cloud generally is:

What if, for some reason, the SAN/servers that store my data are completely damaged? How can I restore my data?

The answer is very simple because Microsoft Azure Storage is durable and supports data replication, therefore you can make sure your storage is highly available.

Replication ensures that your data is copied somewhere else, whether in the same data center, another data center, or even another region.

Microsoft Azure supports multiple options for data replication. You can use whatever you feel suits your business, especially as every type has its own price.

In order to calculate your solution's cost, you can use the Azure Pricing Calculator, which can be reached via the following URL: https://azure.microsoft.com/en-us/pricing/calculator/.

Locally redundant storage (LRS) replicates three copies of your data within the same data center you have your data in. The write requests you do with your storage are not committed until they are replicated to all three copies, which means it replicates synchronously. Not only this, it also makes sure that these three copies exist in different update domains and fault domains. You can revise the terms guide at the beginning of the chapter to understand what the update domain and the fault domain are.

Drawbacks:

• The least durable option, as it replicates only within the same data center
• Your data will be lost if a catastrophic event, such as a volcanic eruption or flood, affects the data center

Advantages:

• It is the cheapest type compared to the other types
• It is the fastest type of data replication, offering the highest throughput since it replicates within the same data center, mitigating the risk of data loss that would occur during data replication caused by a failure having occurred on the original data host
• It is the only available replication type that can be used with Premium Storage at the time of writing

Zone Redundant Storage (ZRS) replicates three copies of data across two or three data centers within one of two regions asynchronously, plus the three copies of data stored within the same data center of the original source of the data.

Drawbacks:

• This type can only be used for Block Blobs (one of the Azure services covered in the next chapter), and a Standard Storage account (general purpose Standard Storage accounts will be covered later in this chapter)
• Does not support metrics or logging
• Does not support conversion for other replication types, such as LRS, GRS, and vice versa
• If a disaster occurs, some data might be lost, because the data replicates to the other data center asynchronously
• If a disaster occurs, there will be some delay in accessing your data until Microsoft failover to the secondary zone

Advantage: It provides higher durability and availability for data than LRS, as it not only replicates in the same data center but also in other data centers.

Geo-redundant storage (GRS) replicates data not only within the same region but also to another region. Firstly, it replicates three copies of data within the same region synchronously, then it replicates another three copies of data to other regions asynchronously.

Drawbacks:

• If a disaster occurs, some data might be lost, because the data replicates to the other regions asynchronously
• If a disaster occurs, there will be some delay in accessing your data until Microsoft initiates failover to the secondary region

Advantages:

• It provides the highest durability and availability, even if a disaster occurs in an entire region
• Unlike ZRS, if the original source of data faces an outage, there will be no possibility of data loss if the other three copies that exist within the same region don't face an outage too, as it replicates synchronously within the same region.

Read-access geo-redundant storage (RA-GRS) follows the same replication mechanism of GRS, in addition, to read access on your replicated data in the other regions.

Drawback: If a disaster occurs, some data might be lost, because the data replicates to the other region asynchronously.

Advantages:

• It provides the highest durability and availability, even if a disaster occurs in a whole region
• If a disaster occurs, you still only have read access to the storage, but no write access until Microsoft initiates failover to the secondary region
• The region with the read access can be used for data retrieval by the nearest offices to it without the need to go to another region to access the data; as a result, the data latency will decrease

As mentioned earlier, Azure provides services for all business types and needs. There are two types based on Azure Storage performance--Standard and Premium.

Standard Storage is the most common type for all the VMs sizes available in Azure. The Standard Storage type stores its data on non-SSD disks. It is commonly used with workloads within which the latency is not critical. Plus, it is low cost and has support for all Azure Storage services (which will be covered in the next chapter). It is also available in all regions.

Premium Storage is designed for low latency applications, such as SQL server, which needs intensive IOPs. Premium Storage is stored on SSD disks, that is why it costs more than Standard Storage. Microsoft recommends using Premium Storage for better performance and higher availability.

Another type of Azure Storage depends on data persistence, which means whether data will be there or not after stopping and starting the VM within which your data exists.

Persistent storage means that the data will be there after stopping and restarting the VM within which your data exists.

Non-persistent storage means that the data will be gone after restarting the VM within which your data exists.

An Azure Storage account is a secure account that provides access to Azure Storage services (which will be covered in the next chapter), and a unique namespace for storage resources.

During the creation of a new Azure Storage account, you will have the option to choose one of two kinds of storage accounts:

• General-purpose storage account
• Blob storage account

A general-purpose storage account gives you access to all Azure Storage services, such as Blobs, Tables, Files, and Queues (these will be covered in the next chapter), and has two performance tiers:

• Standard Storage tier
• Premium Storage tier

Both were covered within the Performance type topic earlier in this chapter.

Unlike a general-purpose storage account, not all Azure Storage services are meant to be stored in a Blob storage account because they are dedicated to storing unstructured data. Therefore, a Blob storage service is the only type allowed to be accessed by a Blob storage account. However, it only supports block and appends Blobs.

A Blob storage account has a usage pattern called access tiers, which determines how frequently you access your data and based on that you will get billed.

Currently, there are two types:

• Hot access tier
• Cool access tier

With the hot access tier, objects will be accessed more frequently, so you will pay less for data access, but pay more for data size.

With the cool access tier, objects will be accessed less frequently, so you will pay more for data access, but less for data size.

The following tips will increase your knowledge about Azure Storage, and will definitely help you when you want to design a storage solution on Azure:

• You cannot switch between an Azure general-purpose storage account and an Azure Blob storage account
• You can switch between access tiers with a Blob storage account, but with the possibility of additional charges being incurred
• A Blob storage account does not support ZRS replication type at the time of writing
• Premium Storage only supports Locally Redundant Storage as a replication type at the time of writing
• Premium Storage is not supported for a Blob storage account at the time of writing
• Azure supports up to 200 storage accounts per subscription by default
• A storage account can store data up to 500 TB
• Azure Storage supports encryption for only two storage services at the time of writing (Blobs and Files), and you can enable it during the storage account creation
• If you are using REST APIs to connect to Azure Storage, you can secure the transfer by enabling that option during the creation of a storage account
• Only lowercase letters and numbers are supported for the name of the storage account

Let's get our hands dirty with creating a storage account with the following parameters:

• Name: packtpubsa
• Deployment model: Resource Manager
• Account kind: General purpose
• Performance: Standard
• Replication: Locally-redundant storage (LRS)
• Storage service encryption (blobs and files): Disabled
• Secure transfer required: Disabled
• Subscription: Select the right subscription for this task
• Resource group: Create a new or select an existing resource group, as per your needs
• Location: Select the nearest location to you

Without further ado, let’s get started:

1. Open the Azure portal from here: https://portal.azure.com/.

2. Click on More services and a new blade will open. In the search bar, write storage account, as shown in the following screenshot:

3. Click on Storage accounts and a new blade will open. Click on Add, as shown in the following screenshot:

4. A new blade will open, wherein you need to fill in the fields and determine the types as per your needs:

5. Fill in the fields as before, and click on Create:

6. Once done, you can find your storage account in the Storage accounts blade:

It is no surprise that we commonly face repetitive and time-consuming tasks. For example, you might want to create multiple storage accounts. You would have to follow the previous guide multiple times to get your job done. This is why Microsoft supports its Azure services with multiple ways of automating most of the tasks that can be implemented in Azure. Throughout this book, two of the automation methods that Azure supports will be used.

PowerShell is commonly used with most Microsoft products, and Azure is no less important than these products.

Mainly, you can use Azure PowerShell cmdlets to manage your Azure Storage, however, you should be aware that Microsoft Azure has two types of cmdlets: one for the classic portal, and another for the portal we are using.

The main difference between the cmdlets of the classic portal and the current portal is there will be an RM added to the cmdlet of the current portal.

For example, if you want to create a storage account in the classic portal, you would use the following cmdlet:

New-AzureStorageAccount

But for the current portal, you would use:

New-AzureRMStorageAccount

By default, you can use Azure PowerShell cmdlets in Windows PowerShell; you will have to install its module first.

There are two ways to install the Azure PowerShell module:

• Download and install the module from the following link: https://www.microsoft.com/web/downloads/platform.aspx
• Install the module from the PowerShell Gallery

1. Open PowerShell in elevated mode.
2. To install the Azure PowerShell module for the current portal, run the Install-Module AzureRM cmdlet. If your PowerShell requires the NuGet provider, you will be asked to agree to install it and you will have to agree to the installation policy modification, as the repository is not available on your environment, as shown in the following screenshot:

1. Log in to your Azure account using the Login-AzureRmAccount cmdlet. You will be prompted to enter your account credentials:

2. Create another storage account with the same properties as we used for the portal, but with a different name:

3. Congratulations! You have created a storage account using PowerShell.

The Azure command-line interface (CLI) is open source, cross-platform, and supports implementing all the tasks you can do in the Azure portal with commands.

Azure CLI comes in two flavors:

• Azure CLI 2.0, which only supports the current Azure portal
• Azure CLI 1.0, which supports both portals

Throughout the book, we will be using Azure CLI 2.0. So, let’s get started with its installation.

To understand what the Azure CLI 2.0 is capable of, we need to install it. Let's do so by following these steps:

1. Download Azure CLI 2.0 from the following link: https://azurecliprod.blob.core.windows.net/msi/azure-cli-2.0.12.msi.
2. Once downloaded, you can start the installation by following the screenshots shown here:
   1. Run the executable files as administrator, and once the wizard opens, click on Install:

1. 2. Once you click on Install, it will start to validate your environment to check whether it is compatible with it or not, then it starts the installation:

1. 3. Once the installation completes, you can click on Finish, and you are good to go:

3. Once done, you can open the cmd and type az to access Azure CLI commands, as shown in the following diagram:

Let's get our hands dirty with the Azure CLI 2.0 to create an Azure Storage account:

1. Log in to your Azure account using the az login command. You have to open the URL that pops up on the CLI and enter the code, as shown in the following screenshot:

2. Create another storage account with the same properties as we used for the portal, but with a different name, as shown in the following screenshot:

So far, we have covered some preliminary subject matters regarding Azure generally, and Azure Storage specifically. Some things were not covered in detail, but detailed discussions will be raised in the coming chapters.

Next, Azure Storage architecture and Azure services will be covered in detail. Therefore, the knowledge gained in this chapter is required for a better understanding of the coming chapter.