AI Risk Management

As organizations increasingly rely on AI for critical decision-making and incorporate it into different areas of their businesses, effective AI risk management should be a top priority. Ensuring the safe and compliant deployment of ML systems is essential to establish trustworthiness in AI solutions. However, many organizations and individuals have very limited understanding of the risks associated with AI systems, often resulting in outcomes that may negatively impact organizations financially or legally. In this chapter, we will explore key AI risk scenarios, highlight the differences between AI risk management and traditional software risk management, and emphasize the importance of having a robust AI risk management practice. We will present a risk management framework that organizations can consider for managing AI risks. Finally, we will discuss how to manage risks at different stages of the ML lifecycle and design ML platforms that support risk management...

Understanding AI risk scenarios

Many of the organizations I have worked with have very limited knowledge about the risks presented in their AI systems. They often treat AI risks the same way they deal with risks associated with traditional software. In reality, AI systems present a new set of risks that we do not normally see in traditional software. With traditional software, the risk is mainly about software vulnerability, a legacy technology stack, malware, misconfiguration, and unauthorized access to data. AI systems are exposed to many of the same software risks; additionally, AI systems can present new kinds of risks such as bias and misinformation. These risks can have significant negative consequences for organizations and individuals that rely on AI systems for business operations and decision-making. AI risks can manifest in many different ways, such as displaying biased behavior or producing unexpected prediction results. Many of the AI risk scenarios are also silent risks...

The regulatory landscape around AI risk management

With the fast advancement of AI technologies and adoption in critical business decision-making, and the negative impacts that AI systems can potentially have on individuals, organizations, and societies, many countries and jurisdictions have established policies, guidance, and regulations to help manage the risks of AI adoption. It is also expected that more and more legislation will be proposed and passed by different countries and jurisdictions at a fast rate.

In the United States (US), the Federal Reserve and the Office of the Comptroller of the Currency (OCC) published the Supervisory Guidance on Model Risk Management (OCC 2011-2012/SR 11-7) as early as 2011. SR 11-7 has become the key regulatory guidance for model risk management in the US. This guidance establishes the main principles for model risk management covering governance, policies and controls, model development, implementation and use, and model validation processes...

Understanding AI risk management

To address the various risks associated with AI and to comply with different compliance regulations, many organizations, especially in the regulated industry, have developed and implemented AI risk management programs. In short, AI risk management is the process of identifying, assessing, and mitigating the risk associated with the use of AI in automated decision-making. The ultimate goal of AI risk management is to establish trust in the AI/ML systems and ensure compliance with applicable rules and regulations.

Trusting an AI system requires rigorous assessment and consideration of the AI system across many different dimensions and criteria. Functionally, a trusted AI system needs to provide valid predictions/responses reliably for its intended use. This means that generated predictions/responses are consistently valid and can be trusted for reliable decision-making. Ethically, a trusted AI system needs to be safe to use, explainable, privacy...

Applying risk management across the AI lifecycle

AI risks can exist in any stage of the AI lifecycle, spanning from business problem identification to the uses of AI systems. In the following sections, we will explore the various risks that can arise at each stage of the AI lifecycle (as illustrated in Figure 12.1) and suggest effective strategies and considerations to mitigate them.

Business problem identification and definition

In this initial stage of the AI lifecycle, organizations develop a comprehensive understanding of the business problems that AI can address. They also outline the overall solution approach and data prerequisites. It is critical during this phase to verify that the AI solution aligns with governance principles, standards, and requirements while achieving specific business objectives.

One significant risk is the regulatory compliance risk, which arises when there is a lack of consideration for potential regulatory requirements. Organizations must...

Designing ML platforms with governance and risk management considerations

ML technology systems play a crucial role in the AI risk management process and activities. To begin with, these systems must be developed and constructed to comply with both internal and external policies and guidelines. Additionally, technology can aid in streamlining and automating ML governance procedures. The following figure illustrates the different ML governance touchpoints in an enterprise ML platform. It is important to know that ML technology alone can only help address a subset of AI risks; other enterprise security technology needs to be incorporated to form a more comprehensive governance and defense mechanism.

Figure 12.2: ML platform and ML governance

In the preceding figure, the ML governance touchpoints have been integrated into the MLOps architecture depicted in Figure 9.4 of Chapter 9, Designing an Enterprise ML Architecture with AWS ML Services.

When an ML platform...

Summary

This chapter delved into several areas related to AI risk management and the technology platforms that support it. By now, you should have a solid understanding of the key AI-related risk scenarios, why AI risk management is critical, and how to detect and address potential risks throughout the AI lifecycle. Additionally, you should be aware of the significance of ML platforms in supporting AI risk management. It is worth noting that AI risk is a vast and complex domain with many unresolved risk challenges and new emergent risks arising rapidly. Moreover, the fast advancement in AI technology and adoption is also creating new risk exposure that risk management professionals must constantly address.

In the next chapter, we will dive deeper into several specific AI risk topics and mitigation techniques, including bias, model explainability, model robustness, and adversarial attacks.

Leave a review!

Enjoying this book? Help readers like you by leaving an Amazon review...