Chapter 7: Doing Automated Machine Learning with Amazon SageMaker Autopilot

"One of the holy grails of machine learning is to automate more and more of the feature engineering process."

– Pedro Domingos

"Automated machine learning, the best thing since sliced bread!"

– Anonymous

Automated Machine Learning (AutoML) via hyperscalers – that is, via cloud providers – has the potential to bring AI democratization to the masses. In the previous chapter, you created a Machine Learning (ML) workflow in SageMaker, and also learned about the internals of SageMaker Autopilot.

In this chapter, we will look at a couple of examples explaining how Amazon SageMaker Autopilot can be used in a visual, as well as in notebook, format.

In this chapter, we will cover the following topics:

• Creating an Amazon SageMaker Autopilot limited experiment
• Creating an AutoML experiment
• Running the SageMaker Autopilot experiment and...

Technical requirements

You will need access to an Amazon SageMaker Studio instance on your machine.

Creating an Amazon SageMaker Autopilot limited experiment

Let's gets a hands-on introduction to applying AutoML using SageMaker Autopilot. We will download and apply AutoML to an open source dataset. Let's get started!

1. From Amazon SageMaker Studio, start a data science notebook by clicking on the Python 3 button, as shown in the following screenshot:

   Figure 7.1 – Amazon SageMaker Launcher main screen

   Download the Bank Marketing dataset from UCI by calling the following URL retrieve commands and save it in your notebook:

   Figure 7.2 – Amazon SageMaker Studio Jupyter Notebook – downloading the dataset

   This Bank Marketing dataset is from a Portuguese banking institution and has the classification goal of predicting the client's subscription to deposit (binary feature, y). The dataset is from Moro, Cortez, and Rita's paper on "A Data-Driven Approach to Predict the Success of Bank Telemarketing. Decision Support Systems", published...

Creating an AutoML experiment

Since the Autopilot data exploration and candidate definition notebooks provide an in-depth overview of the dataset, the complete experiment actually runs these steps and give you a final, tuned model based on the steps described in these notebooks. Now, let's create a full experiment using the same UI as looked at earlier:

1. From Amazon SageMaker Studio, start a data science experiment. Click on the experiment icon in the left-hand pane and create an experiment by providing the experiment name and S3 bucket address, as shown in the following screenshot:

Figure 7.18 – Amazon SageMaker Autopilot – creating the experiment

In the previous Creating an Amazon SageMaker Autopilot limited experiment section, we did the limited run. In this section, we will use the complete experiment feature:

Figure 7.19 – Amazon SageMaker Autopilot – creating the complete experiment...

Running the SageMaker Autopilot experiment and deploying the model

Amazon SageMaker Studio makes it easy for us to build, train, and deploy machine learning models; that is, it enables the data science life cycle. To deploy the model we built in the preceding section, we will need to set certain parameters. For this, you must provide the endpoint name, instance type, how many instances (count), and if you'd like to capture the request and response information. Let's get started:

1. If you select the Data capture option, you will need an S3 bucket for storage, as shown in the following screenshot:

   Figure 7.25 – Amazon SageMaker endpoint deployment

2. Once you've clicked on Deploy, you will see the following screen, which shows the progress of the new endpoint being created:

   Figure 7.26 – Amazon SageMaker endpoint deployment in progress

   Once the deployment is completed, you will see the following status of InService:

   Figure 7.27 – Amazon SageMaker...

Building and running SageMaker Autopilot experiments from the notebook

Customer churn is a real problem for businesses and in this example, we will use our knowledge of completing AutoML in Amazon SageMaker Autopilot to build a customer churn prediction experiment using the notebook. In this experiment, we will use a publicly available dataset of US mobile customers provided by Daniel T. Larose in his book Discovering Knowledge in Data. To demonstrate running the full gamut, the sample notebook executes the Autopilot experiment by performing feature engineering, building a model pipeline (along with any optimal hyperparameters), and deploying the model.

The evolution of the UI/API/CLI paradigm has helped us utilize the same interface in multiple formats; in this case, we will be utilizing the capabilities of Amazon SageMaker Autopilot directly from the notebook. Let's get started:

1. Open the autopilot_customer_churn notebook from the amazon-sagemaker-examples/autopilot...

Summary

Building AutoML systems to democratize AI from scratch is a considerable effort. Therefore, cloud hyperscalers act as enablers and accelerators to jumpstart this journey. In this chapter, you learned how to use Amazon SageMaker Autopilot, both via notebooks and via the experimentation user interface. You were also exposed to the larger AWS machine learning ecosystem and SageMaker's capabilities.

In the next chapter, we will study another major cloud computing platform, Google Cloud Platform, and the AutoML offerings provided it. Happy coding!

Further reading

For more information on the topics that were covered in this chapter, please refer to the following links and resources:

• Mastering Machine Learning on AWS: Advanced machine learning in Python using SageMaker, Apache Spark, and TensorFlow, by Dr. Saket S.R. Mengle , Maximo Gurmendez, Packt Publishing: https://www.amazon.com/Mastering-Machine-Learning-AWS-TensorFlow/dp/1789349796
• Learn Amazon SageMaker: A guide to building, training, and deploying machine learning models for developers and data scientists, by Julien Simon and Francesco Pochetti, Packt Publishing: https://www.amazon.com/Learn-Amazon-SageMaker-developers-scientists/dp/180020891X