Return on Investment Detections

Creating detections and alerts is the bread and butter of any security operations center (SOC) environment. It should not be a surprise to anyone that less than stellar detections are created/triggered daily. This chapter will discuss alerts that we have had the highest efficiency ratings on, as well as the lowest, and how to measure their success. The skills from this chapter will allow you to identify detections that are not efficient, create more efficient alerts, and implement metrics to measure alerts. The topics that we will cover in this chapter include:

• Reviewing examples of poorly created detections and their consequences
• Finding the winners or the best alerts
• Measuring the success of a detection

Technical requirements

For this specific chapter, no installations or specific technologies are required.

Reviewing examples of poorly created detections and their consequences

Detections are created to help fill a security gap, enforce a security policy, and align with a compliance standard, among other reasons. I’d like to say that every detection created is thought through carefully, modeled, and then created with optimization in mind, but that really isn’t the case. The reality is that many alerts are created as a reactive action, so in response to a type of incident or a failed control of an audit. Then, you put in the simplest form of detection and go from there. A lot of the time, you forget about it unless it’s overly noisy with false positives. The detections can have some true positives but usually cause more work than is necessary to weed through the alerts until you get to the point where you have to implement automation or tuning alerts.

The first detection that comes to my mind when I think of poorly created ones was done out of necessity and could...

Finding the winners or the best alerts

Anytime that you have an alert that proves successful and does so a higher percentage of the time (even 30% is exciting), it feels like you have a win. Those are the alert types that have always made me personally excited to triage and you want to find a way to continue that feeling and expand it to other detections. In my opinion, there are a few different categorizations for winning alerts. There is one where the alert is almost always going to be something actionable, whether it just shows a poor security practice, a violation of your acceptable use policy, or something more serious that could lead to an incident investigation starting. Then, there is an alert that is technically a true positive, but there are limited actions that can be taken. Finally, there are the surprise true positive alerts, which leave your team scrambling to triage and put together contextual information.

The first categorization can be all different alerts, but...

Measuring the success of a detection

Measuring success is a key part of any organization, and SOCs, while they are operational, are no different. I’ve run SOC environments in a few different ways; primarily there is a split 60/40 of alert triage to project work. Project work can be anything from creating detections and runbooks to assisting with an audit or ingesting new log sources. Of course, that split is also variable based on incidents. Regardless, finding the right way to measure is key to determining success for both your SOC and detection.

Requirement-setting

The project aspect is straightforward on whether the project is completed or not. For that, I run quarterly sprints where our projects are broken down using business-driven development language or that of Cucumber, an open source software tool that can assist with writing test cases in a broken-down, non-technical format. An example of a ticket would be as follows:

“I’d like to ingest Microsoft...

Summary

Detections and alerting are the bread and butter of any SOC environment, and it’s important that you can determine which ones are successful and which ones need some help. You should look to set up a way to track the efficacy of alerts and audit the detection rules on a rotating basis to ensure they stay up-to-date with your potentially changing environment. After you’ve determined what does and doesn’t work, you need to find a way to tell your story in a quantitative way that will help bring visibility to the risks, successes, SOC environment, need for resources, and so on. The skills gained from this chapter are primarily around key metrics that can be immediately captured within your environment and identifying good and bad detections. In the next chapter, we’ll talk through runbooks and about what to do after an alert is triggered, and discuss the triage process.