Log Aggregation, Risk and Control Monitoring, and Reporting

This is the last chapter of Domain 3: Risk Response and Reporting and is divided into two parts. In the first part, we will look at the different sources for collecting logs, tools, and best practices to aggregate them, and how to analyze those logs. In the second part, we will look at risk and control monitoring, different control assessments, risk and control reporting methods, different key indicators for an executive summary, and the appropriate audience for each.

The aim of this chapter is to learn about the different methods of log sources, aggregation, and analysis. We will also learn about risk and control monitoring, reporting, and how to present reports effectively.

In this chapter, we will cover the following topics:

• Log aggregation and analysis
• Security information and event management
• Risk and control monitoring
• Risk and control reporting
• Key indicators

With that, let us dive...

Log aggregation and analysis

As we learned in the previous chapter, logs play an important role in implementing a detective and corrective control strategy. There are millions and millions of events happening in an organization, and without a proper mechanism to aggregate and analyze these logs, the security team could miss many important events, which could lead to an incident. To understand this, let’s consider the example of Google’s Gmail. Imagine the number of people inserting an incorrect password for their Gmail account at any given time. A human could enter an incorrect password maybe five to seven times in a minute. But what if Gmail encountered that the same person was trying out different passwords 100 times per minute or maybe 1,000 times per minute? This would be impossible for a human, and that’s where this would trigger an alert for Google’s Security Operations Center (SOC). Now, as a SOC analyst, you could review one or two of those attempts...

SIEM

SIEM systems are integrated data correlation tools that help in integrating different systems and collecting, analyzing, and alerting based on intelligent thresholds. SIEM systems can be used to detect malicious events based on pre-defined signatures or behavior. SIEM systems allow risk practitioners to identify risk and bring it to the attention of management before it materializes by correlating it with similar events.

An important consideration for implementing a SIEM system is the regular monitoring and fine-tuning of alerting rules to reduce false-positive alerts as much as possible. This constant fine-tuning ensures that the risk practitioner is only focusing on important alerts that require action.

A SIEM system can translate all the logs in management reports and dashboards that are important for management reporting, can support compliance requirements for log retention, and will be helpful for forensic analysis. The risk manager should ensure that the logs are...

Risk and control monitoring

The risks to an organization are ever-changing, and so is the risk profile. Risks encountered a year before may not be relevant anymore, and the controls recently implemented for the latest risk may have already become outdated. A risk practitioner should continuously monitor, benchmark, and improve the control environment to meet organizational objectives. The monitoring of controls can be done through self-assessments or independent third-party audits. Exceptions to controls should be reported, followed up, and addressed with corrective actions. In the following section, we will review some techniques that the risk practitioner can implement with the help of risk owners for effective risk and control monitoring.

Types of control assessments

Before we jump into the techniques for control assessment, let’s briefly review what this term means. Control assessment is the process of evaluating and examining the effectiveness and adequacy of internal...

Risk and control reporting

In the previous section, we reviewed the importance of risk monitoring and how it can impact an organization’s resilience toward malicious attacks. In this section, we will review how those monitored risks and metrics can be best reported to the management team. Different organizations choose different mechanisms to report on risks and controls. Some are okay with sending a brief executive summary, while others need to elaborate with reports and dashboards. There are no right or wrong ways to present these risks to senior management; however, the risk practitioner, as well as the business owner, should tailor the reports and reporting mechanism per the audience. Would it really make any sense to report the number of phishing attempts in the past month to the head of physical security?

Here are some key aspects the risk practitioner should keep in mind while reporting:

• Audience: Who is the right audience for the report?
• Actionability...

Key indicators

The metrics reported in dashboards are called key indicators. Three types of indicators are reported in a dashboard:

• Key performance indicators (KPIs): KPIs are used to understand and enable the measurement of control performance. The level of performance can be different for each organization, and therefore, the risk manager should strive to define KPIs that make the most sense to the organization’s objectives and risk appetite. An example of a KPI could be a reduction in phishing emails after implementing a new tool.
• Key risk indicators (KRIs): KRIs are considered to be highly probable indicators designed to predict risks that could breach the defined thresholds. The goal of defining KRIs is to monitor and analyze trends, determine the effectiveness and efficiency of controls to make informed decisions for current controls and planned countermeasures, and alert relevant stakeholders when the risk breaches the predefined thresholds. An example of...

Summary

At the beginning of this chapter, we learned about log aggregation and analysis, risk and control monitoring, reporting, and key indicators. We then learned about the importance of effectively collecting and analyzing logs, implementing SIEM systems, continuously monitoring and improving control environments, and tailoring risk reports to the audience. In the following section, we reviewed the importance of various key indicators, such as KPIs, KRIs, and KCIs, for measuring control performance, predicting risks, and assessing control effectiveness. In the final section, we gained insights into managing risks and reporting them in a meaningful and actionable manner.

In the next chapter, we will delve deeper into enterprise architecture and information technology.

Review questions

1. What is the primary purpose of log aggregation in an organization?
   1. To identify and analyze important events and potential incidents
   2. To synchronize the clocks of different systems for accurate logging
   3. To eliminate the need for logging across multiple systems
   4. To provide a centralized database for log storage
2. Which of the following is a commonly used tool for log aggregation and analysis?
   1. SIEM system
   2. Vulnerability assessment tool
   3. Penetration testing tool
   4. Heat map generator
3. What is the role of KRIs in risk monitoring?
   1. To predict risks that may breach predefined thresholds
   2. To measure the effectiveness of controls
   3. To indicate weaknesses in control implementation
   4. To assess control performance against goals
4. Which format of risk reporting is known for its flexibility in combining qualitative and quantitative metrics?
   1. Executive summary
   2. Heat map
   3. Scorecard
   4. Dashboard
5. What are the key characteristics of a SMART metric for selecting key indicators?
   1. Specific, Measurable, Attainable...

Answers

1. A. Log aggregation helps in collecting and analyzing logs to identify significant events and potential incidents that could impact the organization’s security.
2. A. SIEM systems are commonly used for log aggregation and analysis, providing a centralized platform to collect, analyze, and correlate logs from various sources.
3. A. KRIs are designed to serve as highly probable indicators that predict risks breaching predefined thresholds, helping in proactive risk management.
4. D. Dashboards are known for their flexibility in combining qualitative and quantitative metrics, providing a comprehensive and visual representation of key risk and control information.
5. A. SMART metrics are essential characteristics for selecting key indicators. They should be specific, measurable, attainable, relevant to goals, and time-bound.
6. D. Penetration testing involves simulating real attacks to identify vulnerabilities that have not yet been discovered, providing valuable...