Chapter 10: Azure Data Explorer Security

As recently as 6 years ago, the emphasis on public cloud security was not what it is today. I remember that, at one point, disk encryption was not available on storage accounts. Then, it became an option that could be enabled, and today, it is enabled by default.

One of the biggest concerns with the public cloud is ensuring that our data and resources are not accessible to just about anyone on the internet. In this chapter, we will learn how to secure our Azure Data Explorer (ADX) instances using identity management and how to perform network filtering on unwanted traffic and bad actors.

We will begin by introducing some of the basic terminology and concepts you should be familiar with, such as the principle of least privilege and role-based access control (RBAC). Next, we will explore the concepts of identity management with Azure Active Directory (AAD). We will also learn about the differences between security principals, users, groups...

Technical requirements

The code examples for this chapter can be found in the Chapter10 folder of this book's GitHub repository: https://github.com/PacktPublishing/Scalable-Data-Analytics-with-Azure-Data-Explorer.git.

Introducing identity management

When you signed up to Azure earlier in this book, an AAD tenant was created. AAD is Microsoft's cloud-based identity and access management service and is used by other major services such as Office365. AAD is a great service that allows you to manage your users and devices, supports multi-factor authentication (MFA) and privileged identity management (PIM), and so on. I would not be doing AAD justice if I tried to cover everything about AAD here. If you would like to learn more, I recommend Packt Publishing's Mastering Identity and Access Management with Microsoft Azure – Second Edition.

Before learning how to manage users, it is important to understand RBAC and the differences between the management plane and the data plane.

Introducing RBAC and the management and data planes

Before we introduce RBAC, it is important to understand what authentication and authorization are and how they differ from one another. Authentication...

Introducing virtual networking and subnet delegation

As we saw in the previous section, identity management is a good method for restricting access to ADX clusters. We can control access at both the management and data plane levels, but our cluster is still available on the public internet. Anyone who knows the name of our cluster could potentially connect by guessing usernames and passwords.

Like a lot of Azure resources, such as storage accounts and Azure SQL, they are accessible on the internet by default. The problem with this default deployment is that we cannot restrict inbound and outbound traffic. Azure supports advanced deployments that allow us to deploy resources within a virtual network. Virtual networks let us create private networks on Azure to isolate and restrict access to our resources, such as virtual machines, ADX clusters, and so on.

Deploying our ADX cluster in a virtual network gives us more control over inbound and outbound traffic. We can use NSGs to...

Filtering traffic with NSGs

When we deployed the ADX cluster, the deployment also updated our route table and NSG. The route table was updated to route specific cluster traffic to the internet, as shown in the following screenshot:

Figure 10.21 – User-defined routing

These routes ensure our cluster can still communicate with Azure Monitor and ADX cluster management.

In the next section, we will learn what they are, how they work, and why we should use them. Once we understand the theory, we will update our NSG by adding a new security rule to allow traffic to our ADX cluster.

Introducing NSGs

Before we add any security rules to our NSG, let's spend some time discussing what they are, how they work, and why we should use them.

NSGs are one of the fundamental security-related building blocks in Azure. NSGs allow us to filter both inbound and outbound traffic in an Azure Virtual Network. NSGs can be assigned to subnets and Network...

Summary

In this chapter, we learned about the importance of security, especially on the public cloud, and introduced some of the basic terminology and concepts you should be familiar with, such as the principle of least privilege and RBAC. Next, we introduced the concepts of identity management with AAD, explained the differences between security principals, users, groups, and service principals, and the different levels of access, such as the management plane versus the data plane.

Next, we demonstrated how to restrict access to your ADX cluster using virtual networks and subnet delegation.

Finally, we introduced NSGs and explained how to use them to filter inbound traffic. Then, we demonstrated how to route inbound traffic from your public IP.

In the next chapter, you will discover how to troubleshoot performance issues with queries and learn about the best practices for writing queries and managing your clusters.

Questions

Before moving on to the next chapter, test your knowledge by trying out these exercises. The answers can be found at the back of this book:

1. Assign the contributor role to one of your AAD users in the management plane.
2. Assign the Database ingestor role to a user.
3. What happens when you log in to the ADX Web UI and try to query the database as that user?