Chapter 9: Testing and Securing Your ML Solution

In this chapter, we will delve into Machine Learning (ML) solution testing and security aspects. You can expect to get a primer on various types of tests to test the robustness and scalability of your ML solution, as well as the knowledge required to secure your ML solution. We will look into multiple attacks on ML solutions and ways to defend your ML solution.

In this chapter, we will be learning with examples as we perform load testing and security testing for the business use case of weather prediction we have been previously working on. We will start by reflecting on the need for testing and securing your ML solution and go on to explore the other following topics in the chapter:

• Understanding the need for testing and securing your ML application
• Testing your ML solution by design
• Securing your ML solution by design

Understanding the need for testing and securing your ML application

The growing adoption of data-driven and ML-based solutions is causing businesses to have to handle growing workloads, exposing them to extra levels of complexities and vulnerabilities.

Cybersecurity is the most alarming risk for AI developers and adopters. According to a survey released by Deloitte (https://www2.deloitte.com/us/en/insights/focus/cognitive-technologies/state-of-ai-and-intelligent-automation-in-business-survey.html), in July 2020, 62% of adopters saw cybersecurity risks as a significant or extreme threat, but only 39% said they felt prepared to address those risks.

In this section, we will look into the need for securing ML-based systems and solutions. We will reflect on some of the broader challenges of ML systems such as bias, ethics, and explainability. We will also study some of the challenges present at each stage of the ML life cycle relating to confidentiality, integrity, and availability...

Testing your ML solution by design

On top of performing regular software development tests, such as unit tests, integration tests, system testing, and acceptance testing, ML solutions need additional tests because data and ML models are involved. Both the data and models change dynamically over time. Here are some concepts for testing by design; applying them to your use cases can ensure robust ML solutions are produced as a result.

Data testing

The goal of testing data is to ensure that the data is of a high enough quality for ML model training. The better the quality of the data, the better the models trained for the given tasks. So how do we assess the quality of data? It can be done by inspecting the following five factors of the data:

• Accuracy
• Completeness (no missing values)
• Consistency (in terms of expected data format and volume)
• Relevance (data should meet the intended need and requirements)
• Timeliness (the latest or up-to-date data)

Based...

Hands-on deployment and inference testing (a business use case)

When you have your service (either API or ML) ready and you are about to serve it to the users but you don't have any clue about how many users it can actually handle and how it will react when many users access it simultaneously, that's where load testing is useful to benchmark how many users your service can serve and to validate whether the service can cater to the business requirements.

We will perform load testing for the service we deployed previously (in Chapter 7, Building Robust CI and CD Pipelines). Locust.io will be used for load testing. locust.io is an open source load-testing tool. For this, we will install locust (using pip) and curate a Python script using the locust.io SDK to test an endpoint. Let's get started by installing locust:

1. Install locust: Go to your terminal and execute the following command:

   pip install locust

   Using pip, locust will be installed – it takes around...

Securing your ML solution by design

Securing your ML applications is more important than ever due to the growing adoption of AI to provide smart applications. Designing and developing ML systems without keeping security in mind can be costly in terms of exposing the system to hackers, leading to manipulation, data breaches, and non-compliance. Robustness and security play an important role in ensuring an AI system is trustworthy. To build trustworthy ML applications, keeping security in mind is vital to not leave any stones unturned.

Figure 9.8 shows a framework for creating secure ML applications by design. The framework addresses key areas in the ML life cycle, ensuring confidentiality, integrity, and availability within those specific stages. Let's reflect upon each area of the ML life cycle and address the issues of confidentiality, integrity, and availability in each area:

Figure 9.8 – Framework for securing the ML life cycle by design

Let...

Summary

In this chapter, we have learned the key principles of testing and security by design. We explored the various methods to test ML solutions in order to secure them. For a comprehensive understanding and hands-on experience, implementation was done to load test our previously deployed ML model (from Chapter 7, Building Robust CI and CD Pipelines) to predict the weather. With this, you are ready to handle the diverse testing and security scenarios that will be channeled your way.

In the next chapter, we will delve into the secrets of deploying and maintaining robust ML services in production. This will enable you to deploy robust ML solutions in production. Let's delve into it.