Chapter 8: Choosing Real-Time versus Batch Scoring

As you have experienced in the previous chapters, training AutoML models is simple and straightforward. Whether you choose to train a model using the Azure Machine Learning Studio (AMLS) GUI or code an AutoML solution in Python using Jupyter, you can build highly accurate machine learning (ML) models in minutes. However, you still need to learn how to deploy them. In Azure, there are two main ways you can deploy a previously trained ML model to score new data: real-time and batch.

In this chapter, you will begin by learning what a batch scoring solution is, when to use it, and when it makes sense to retrain batch models. Continuing, you will learn what a real-time scoring solution is, when to use it, and when it makes sense to retrain real-time models. Finally, you will conclude by reading a variety of different scenarios and determining which type of scoring you should use. All scenarios are based on common problems faced by real...

Technical requirements

Chapter 7, Using the Many Models Solution Accelerator, featured a lot of heavy Python coding. This chapter is a bit of a reprieve; you will not be coding, but you will be learning important skills through reading business scenarios and applying the proper solutions. As such, there are no technical requirements in this chapter.

Architecting batch scoring solutions

Batch inferencing refers to scoring new data points in batches on a recurring time-based schedule. New data is collected over time and subsequently scored, generating new predictions. This is the most common way modern companies use ML models.

In this section, you will learn how to architect a complete, end-to-end batch scoring solution using Azure AutoML-trained models. You will also learn why, and in what situations, you should prioritize batch scoring over real-time scoring solutions.

Understanding the five-step batch scoring process

Each batch scoring solution you make should follow a five-step process. This process begins by training and registering an ML model as you did in the previous chapters using AMLS. Regression, classification, and forecasting models all follow the same pattern. In order, the five steps are as follows:

1. Train a model. You can train a model either using the AMLS GUI as you did in Chapter 3, Training...

Architecting real-time scoring solutions

Real-time inferencing refers to scoring new data points as they arrive instead of on a time-based schedule. New data flows in, new predictions come out. While not as common as batch inferencing, real-time inferencing is used by companies in a number of scenarios such as credit card fraud detection, anomaly detection on the factory floor, and recommending products when you're online shopping.

In this section, you will learn how to architect a complete, end-to-end real-time scoring solution using Azure AutoML-trained models. You will also learn why, and in what situations, you should prioritize real-time scoring over batch scoring solutions.

Understanding the four-step real-time scoring process

Real-time scoring solutions follow a slightly different process than batch scoring solutions. There are only four steps. Like batch solutions, the process begins by training an ML model and registering it as you did in previous chapters....

Determining batch versus real-time scoring scenarios

When confronted with real business use cases, it is often difficult to distinguish how you should deploy your ML model. Many data scientists make the mistake of implementing a batch solution when a real-time solution is required, while others implement real-time solutions even when a cheaper batch solution would be sufficient.

In the following sections, you will look at different problem scenarios and solutions. Read each of the six scenarios and determine whether you should implement a real-time or batch inferencing solution. First, you will look at every scenario. Then, you will read each answer along with an explanation.

Scenarios for real-time or batch scoring

In this section, you are presented with six scenarios. Read each carefully and decide whether a batch or real-time scoring solution is most appropriate.

Scenario 1 – Demand forecasting

A fast-food company is trying to determine how many bags of frozen...

Summary

You now have a firm understanding of batch and real-time inferencing, and when to use which type of scoring solution. This is important, as even seasoned data scientists occasionally make mistakes when designing end-to-end ML solutions.

Furthermore, most ML courses focus on training models instead of deploying them, but to be an effective data scientist, you must be proficient at both. In the upcoming chapters, you will learn how to code each of these inferencing methods in AMLS.

In Chapter 9, Implementing a Batch Scoring Solution, you will learn step by step how to use the ML models you've already built in batch scoring scenarios. You will create ML pipelines in AMLS and learn how to schedule them to run on a timer. This will allow you to easily productionalize your ML models and become a valuable asset to your company or organization.