Azure Storage Security and Additional Tooling

In the previous two chapters, we talked about Azure Storage with a focus on Blob and File Storage Services. Those chapters were meant to give you a general overview of those services’ capabilities and configuration options. This chapter will summarize our findings and lessons learned and, additionally, give you more insights into security options for Azure Storage and what’s possible to do and implement when challenged with automation tasks and data import/export activities.

Topics covered in this chapter may be considered advanced, as they require prior knowledge of Blob Service configuration (described in Chapter 9), Azure AD concepts (Chapter 2), and managed identities (Chapter 2). Let’s see what’s going to be described shortly:

• Configuring soft delete in Blob Storage
• Using Azure AD for authorization
• Using managed identities
• Considering various options for automation
• Using the Azure...

Technical requirements

For the exercises in this chapter, you’ll need the following:

• The Azure CLI (https://learn.microsoft.com/en-us/cli/azure/install-azure-cli)
• A command line of any kind (either the one available in your operating system or your favorite one)

The Code in Action video for this book can be viewed at: https://packt.link/GTX9F

Configuring soft delete in Blob Storage

Soft delete as a concept has already been briefly described in Chapter 9 in the Configuring soft delete section. Let’s quickly remind ourselves what it means to have soft delete configured for a service. Under normal circumstances, any time you delete data from a service (storage or database), it’ll be permanently deleted unless it’s part of a backup or snapshot of your data. While backups could be described as a kind of soft delete solution, they’re rather meant for disaster recovery scenarios. Soft delete, on the other hand, is a simple feature that marks your data as deleted without it being deleted permanently. This allows you to restore data at any time without the need to recover a whole storage account or database.

Soft delete will work differently for different services, as it’s a feature that is tightly coupled with its functionalities and capabilities. For instance, in File Service, you’re...

Using Azure AD for authorization

When interacting with Azure Storage, you may use three different options when it comes to authenticating and authorizing each request:

• Access key
• Shared access token
• Azure AD token

While the former two are the native ways of connecting with Azure Storage and are the simplest options, Azure AD authentication and authorization is a feature that gives you the most flexibility and improved security. Let’s discuss it further to have a better understanding of its capabilities.

Azure AD authorization benefits

Both access keys and shared access tokens are simple mechanisms for securing access to Azure Storage. The access flow for them looks like this, starting with the access key:

1. Obtain the access key from your Azure Storage account.
2. Share the key with a client.

Now, each client we share a key with can use it to connect to our storage account. What’s more, it gives them full access to all services...

Using managed identities

When deploying services to Azure, you can utilize a variety of different resources that will eventually interact with each other. You can have a virtual machine hosting an application connecting to a database, data orchestrators interacting with storage services, and many, many other possibilities. When designing such solutions, one often thinks about authentication and authorization mechanisms that will be used to determine what one service can and cannot do when connecting to another one. Most of the time, this can be solved by using concepts such as connection strings, access policies, or an in-built RBAC mechanism. This, however, is not always an ideal way of handling that challenge. In this section, we’ll talk more about using managed identities when connecting with Azure Storage to see how they make the whole setup simple.

Definition of managed identity

A managed identity in Azure is a concept for assigning an identity to a service that normally...

Considering various options for automation

Azure Storage, as a generic solution for building solutions that need scalable and reliable storage, can be easily automated using either native or external tools. In this section, we’ll talk about scenarios that could possibly be automated and how to approach this. Let’s start with managing data stored in Blob Service.

Automated cleanup of data

One of the biggest challenges when working with data-based infrastructure is providing a way to automatically clean up data that is no longer needed. This is often a requirement in systems that must follow legal requirements regarding how long users’ data is stored. In Azure Storage, if you choose to use Blob Storage as your storage service, you can leverage life cycle management, which will help you move files between tiers (hot -> cool -> archive) and delete them after a certain period of inactivity.

SAS token handling

If you decide to use SAS tokens as your...

Using the Azure Import/Export service

When deciding to migrate data to or from Azure, you always need to consider the volume of files you need to import or export. While incoming traffic (inbound) is free in Azure, outbound connectivity is not. This means that transferring large volumes of data from an Azure data center may be a costly operation unless it’s planned accordingly.

Important note

The price for each GB of data downloaded from an Azure data center depends on various factors. See the following documentation for a better understanding of the topic: https://azure.microsoft.com/en-us/pricing/details/bandwidth/.

Besides pricing, migrating large volumes of data gives us one more thing to consider – the time needed to complete an operation. If we’re talking about transferring hundreds of terabytes or even petabytes, you need to ensure that your network can handle the load in an acceptable time. As you’re not always in charge of the network infrastructure...

Summary

This chapter was focused on extending your knowledge regarding Azure Storage and refining it based on the topics we covered initially in Chapters 9 and 10. We talked about soft delete and versioning of blobs in Blob Storage and discussed in detail how different authorization methods for Azure Storage may work in various scenarios. We also looked at managed identities in the context of Azure Storage, so you can configure services to interact with tables, blobs, or queues using an identity defined in Azure AD. In the remaining parts, we revisited automation options for Azure Storage and importing/exporting data using physical disks that can be shipped to an Azure data center.

Lessons from this chapter will help you work with Azure Storage in advanced scenarios, which include improving the reliability of data stored in the service, improving security, and implementing automation. As Azure Storage is one of the most popular services in Azure, becoming proficient in it will help...