Model Deployment

In the previous chapter, you explored various aspects of Amazon SageMaker, including different instances, data preparation in Jupyter Notebook, model training with built-in algorithms, and crafting custom code for training and inference. Now, your focus shifts to diverse model deployment choices using AWS services.

If you are navigating the landscape of model deployment on AWS, understanding the options is crucial. One standout service is Amazon SageMaker – a fully managed solution that streamlines the entire machine learning (ML) life cycle, especially when it comes to deploying models. There are several factors that influence the model deployment options. As you go ahead in this chapter you will learn different options to deploy models using SageMaker.

Factors influencing model deployment options

Here are the primary factors that play a crucial role in determining a model deployment option:

• Scalability requirements
  • High traffic: Imagine you are developing a recommendation system for a popular e-commerce platform expecting fluctuating traffic throughout the day. If the application anticipates high traffic and varying loads, services such as Amazon SageMaker with autoscaling capabilities or AWS Lambda may be preferable. This is crucial to maintain performance during peak hours.
• Real-time versus batch inference
  • Real-time inference: Consider a fraud detection system for a financial institution where immediate decisions are essential for transaction approval or denial. For such real-time predictions, services such as Amazon SageMaker and AWS Lambda are suitable. For fraud detection, these services provide low-latency responses, enabling quick decisions on the legitimacy of transactions. Real-time transactions trigger immediate...

SageMaker deployment options

Amazon SageMaker offers diverse deployment options to deploy ML models effectively. In this section, you will explore different ways of deploying models using SageMaker, providing technology solutions with scenarios and examples.

Real-time endpoint deployment

In this scenario, you have a trained image classification model, and you want to deploy it to provide real-time predictions for incoming images.

Solution

Create a SageMaker model and deploy it to a real-time endpoint.

Steps

1. Train your model using SageMaker training jobs.
2. Create a SageMaker model from the trained model artifacts.
3. Deploy the model to a real-time endpoint.

Example code snippet

from sagemaker import get_execution_role

from sagemaker.model import Model

from sagemaker.predictor import RealTimePredictor

role = get_execution_role()

model_artifact='s3://your-s3-bucket/path/to/model.tar.gz'

model = Model(model_data=model_artifact...

Creating alternative pipelines with Lambda Functions

Indeed, SageMaker is an awesome platform that you can use to create training and inference pipelines. However, you can always work with different services to come up with similar solutions. One of these services, which you will learn about next, is known as Lambda functions.

AWS Lambda is a serverless compute service where you can run a function as a service. In other words, you can concentrate your efforts on just writing your function. Then, you just need to tell AWS how to run it (that is, the environment and resource configurations), so all the necessary resources will be provisioned to run your code and then discontinued once it is completed.

Throughout Chapter 3, AWS Services for Data Migration and Processing, you explored how Lambda functions integrate with many different services, such as Kinesis and AWS Batch. Indeed, AWS did a very good job of integrating Lambda with 140+ services (and the list is constantly increasing...

Working with step functions

Step Functions is an AWS service that allows you to create workflows to orchestrate the execution of Lambda functions. This is so that you can connect them in a sort of event sequence, known as steps. These steps are grouped in a state machine.

Step Functions incorporates retry functionality so that you can configure your pipeline to proceed only after a particular step has succeeded. The way you set these retry configurations is by creating a retry policy.

Important note

Just like the majority of AWS services, AWS Step Functions also integrates with other services, not only AWS Lambda.

Creating a state machine is relatively simple. All you have to do is navigate to the AWS Step Functions console and then create a new state machine. On the Create state machine page, you can specify whether you want to create your state machine from scratch or from a template, or whether you just want to run a sample project.

AWS will help you with this state...

Scaling applications with SageMaker deployment and AWS Autoscaling

Autoscaling is a crucial aspect of deploying ML models in production environments, ensuring that applications can handle varying workloads efficiently. Amazon SageMaker, combined with AWS Auto Scaling, provides a robust solution for automatically adjusting resources based on demand. In this section, you will explore different scenarios where autoscaling is essential and how to achieve it, using SageMaker model deployment options and AWS Auto Scaling.

Scenario 1 – Fluctuating inference workloads

In a retail application, the number of users making product recommendation requests can vary throughout the day, with peak loads during specific hours.

Autoscaling solution

Implement autoscaling for SageMaker real-time endpoints to dynamically adjust the number of instances, based on the inference request rate.

Steps

1. Configure the SageMaker endpoint to use autoscaling.
2. Set up minimum and maximum...

Securing SageMaker applications

As ML applications become integral to business operations, securing AWS SageMaker applications is paramount to safeguard sensitive data, maintain regulatory compliance, and prevent unauthorized access. In this section, you will first dive into the reasons for securing SageMaker applications and then explore different strategies to achieve security:

• Reasons to secure SageMaker applications
  • Data protection: ML models trained on sensitive data, such as customer information or financial records, pose a significant security risk if not adequately protected. Securing SageMaker ensures that data confidentiality and integrity are maintained throughout the ML life cycle.
  • Compliance requirements: Industries such as healthcare and finance are subject to stringent data protection regulations. Securing SageMaker helps organizations comply with standards such as the Health Insurance Portability and Accountability Act (HIPAA) or the General Data Protection Regulation...

Summary

In this chapter, you dived into deploying ML models with Amazon SageMaker, exploring factors influencing deployment options. You looked at real-world scenarios and dissected them to try out hands-on solutions and code snippets for diverse use cases. You emphasized the crucial integration of SageMaker deployment with AWS Auto Scaling, dynamically adjusting resources based on workload variations. You focused on securing SageMaker applications, presenting practical strategies such as VPC endpoints, IAM roles, and encryption practices. Referring to the AWS documentation for clarifying any doubts is also the best option. It is always important to design your solutions in a cost-effective way, so exploring the cost-effective way to use these services is equally important as building the solution.

Exam Readiness Drill – Chapter Review Questions

Apart from a solid understanding of key concepts, being able to think quickly under time pressure is a skill that will help you ace your certification exam. That is why working on these skills early on in your learning journey is key.

Chapter review questions are designed to improve your test-taking skills progressively with each chapter you learn and review your understanding of key concepts in the chapter at the same time. You’ll find these at the end of each chapter.

How To Access These Resources

To learn how to access these resources, head over to the chapter titled Chapter 11, Accessing the Online Practice Resources.

To open the Chapter Review Questions for this chapter, perform the following steps:

1. Click the link – https://packt.link/MLSC01E2_CH10.

   Alternatively, you can scan the following QR code (Figure 10.6):

Figure 10.6 – QR code that opens Chapter...

Working On Timing

Target: Your aim is to keep the score the same while trying to answer these questions as quickly as possible. Here’s an example of how your next attempts should look like:

Attempt	Score	Time Taken
Attempt 5	77%	21 mins 30 seconds
Attempt 6	78%	18 mins 34 seconds
Attempt 7	76%	14 mins 44 seconds

Table 10.1 – Sample timing practice drills on the online platform

Note

The time limits shown in the above table are just examples. Set your own time limits with each attempt based on the time limit of the quiz on the website.

With each new attempt, your score should stay above 75% while your “time taken...

The rest of the chapter is locked