Logging, Monitoring, and Threat Detection

Following best practices is not enough. The threat landscape changes every day and adversaries find new ways to gain access to our resources. Monitoring the safeguards we have put in place is vital to maintaining our security posture. In this chapter, we will see how to monitor our resources and see how effective our security measures are in preventing and detecting threats. We will learn how Azure Monitor works and how to configure logging, retention, and notifications. Finally, we will explore some features of Defender for Cloud and Microsoft Sentinel that can further help us protect our resources and mitigate threats even in real time.

In this chapter, we’re going to cover the following main topics:

• Enabling logging and configuring data retention for Azure services
• Securing resources with Microsoft Defender
• Exploring threat management with Sentinel

By the end of this chapter, we will be able to set up alerts...

Technical requirements

Although this chapter deals mostly with monitoring and logging, knowing the Kusto Query Language (KQL) might come in handy when implementing solutions.

KQL is a query language used to query, analyze, and visualize large datasets stored in Azure Data Explorer, Azure Monitor, Microsoft Sentinel, and Application Insights. KQL is a powerful language that allows you to perform various operations on your data, including filtering, aggregating, joining, and visualizing it.

KQL learning resources

Some resources to learn KQL can be found here: https://learn.microsoft.com/en-us/azure/data-explorer/kql-learning-resources.

Enabling logging and configuring data retention for Azure services

As soon as we create an Azure subscription, we get full monitoring capabilities with the Azure Monitor service. This is a service where we do not need to enable or do any action, it is automatically available for our subscriptions. Although it provides us with full stack monitoring and advanced analytics, there are different things we can do with services that work on top of Azure Monitor. Azure Monitor can monitor and combine data in Azure, on-premises, and other clouds.

You can access the Azure Monitor service by searching for monitor on the search bar and clicking on the resource. In this blade, you will see all Monitor has to offer:

Figure 9.1 – An Azure Monitor overview

Let us see the key components of Azure Monitor.

Working with Azure Monitor

There are two types of data Monitor gathers, metrics and logs. Metrics are numerical values that represent various aspects of...

Securing resources with Microsoft Defender

Microsoft Defender for Cloud is a cloud-native application protection platform (CNAPP) designed specifically for cloud environments. It offers a comprehensive set of security measures and best practices, aimed at safeguarding cloud-based applications against a wide range of cyber attacks and vulnerabilities. Microsoft Defender for Cloud combines several functionalities, including a cloud workload protection platform (CWPP) focusing on infrastructure, storage, and so on, a cloud security posture management (CSPM) solution to prevent security issues, and a DevSecOps solution that helps to secure code across different clouds if needed. Defender includes a basic CSPM without additional cost. There are advanced features that you can enable on top of that, including attack path analysis, the cloud security explorer, advanced threat hunting, security governance, as well as tools to evaluate your security compliance across regulatory standards that...

Exploring threat management with Sentinel

Microsoft Sentinel is a cloud-native Security Information and Event Management (SIEM) and Security Orchestration, Automation, and Response (SOAR) solution. It offers intelligent security analytics and threat intelligence centrally for Azure and other clouds. With Sentinel, we have smart alert detection, threat visibility, hunting, and response, all in a single pane. There are several benefits to using Sentinel for the aforementioned tasks:

• As a cloud solution, it scales with our data, and we pay for what we use.
• Microsoft Sentinel gathers data using connectors from a wide range of sources, including Azure services, on-premises environments, and other clouds.
• The service comes with built-in ML models that help to identify suspicious activities and reduce false positives. Over time, these models can be trained to improve their accuracy based on your organization’s unique patterns.
• Threat hunting is done using KQL to...

Summary

In this chapter, we learned to utilize multiple services, ensuring we can monitor our resources effectively by enabling different services and learning how we can start to use the logs we gather to prevent security incidents. The first one we worked with was Azure Monitor, using Monitor alerts to make sure we can be notified about any issues. By combining the capabilities of Monitor Log Analytics and Application Insights, we can have end-to-end monitoring of our resources and our model endpoints. Additionally, by using Microsoft Defender for Cloud, we can get recommendations to implement best practices, and we can use Microsoft Sentinel for advanced threat management. Now that we have a comprehensive view of the best practices across different surface areas included in a ML project, we can combine them and see how we can build a security baseline for our Azure resources in the next chapter.