Chapter 6: Machine Learning with AWS

"Whatever you are studying right now, if you are not getting up to speed on deep learning, neural networks, etc., you lose. We are going through the process where software will automate software, automation will automate automation."

– Mark Cuban

In the previous chapter, you were introduced to the Azure Machine Learning (ML) landscape and how to do automated ML in the Azure platform. In this chapter, you will learn how to get started with ML using Amazon Web Services (AWS), along with different offerings and a detailed understanding of the ginormous AWS cloud stack.

The theme of this chapter is to get started with an introduction to AWS ML capabilities to give a wider perspective of this large ecosystem; not only AWS as a hyperscaler but also the breadth of the field itself. Many use cases and permutations require specialized solutions and there is no one-size-fits-all solution for an enterprise's AI and ML needs...

ML in the AWS landscape

Gartner is among a few major advisory companies that regularly review the landscape of technology and provide a comprehensive overview of their findings in their Magic Quadrant reports. In its latest release, the Magic Quadrant contains Anaconda and Altair as niche players, Microsoft, DataRobot, KNIME, Google, H2O.ai, RapidMiner, and Domino as visionaries, IBM as a challenger, and Alteryx, SAS, Databricks, MathWorks, TIBCO, and Dataiku as leaders in the data science and ML space.

It is surprising for us to not see AWS mentioned here. There are six companies in the leadership quadrant due to their consistent record of data science and AI solution deliveries, and seven are classified as visionaries. However, AWS not making it to the visionaries and/or the leaders quadrant is attributed to the announcement delay. The AWS flagship AI products SageMaker Studio and SageMaker Autopilot were announced after the deadline for Gartner submission; hence they didn&apos...

Getting started with AWS ML

In this section, we will do a walkthrough of the AWS Management Console and show you how to use AWS SageMaker with step-by-step instructions. Let's dive in. The AWS ML environment is fairly intuitive and easy to work with:

1. To start, first open up the AWS Management Console by visiting aws.amazon.com in your browser. Now, click on Sign in to the Console, or log back in (if you are a returning user):

   Figure 6.5 – AWS Management Console

2. Enter your root (account) user's email address in the Root user email address field to proceed:

   Figure 6.6 – AWS Management Console login

3. Upon successful login, you will be taken to the following screen, the AWS Management Console:

   Figure 6.7 – AWS Management Console

4. AWS has a collection of tons of different services. In the AWS Management Console, find the services search box, then type sagemaker to find the Amazon SageMaker service, as shown in the following screenshot, and...

AWS SageMaker Autopilot

SageMaker Autopilot, as the name suggests, is a fully managed system that provides an automatic ML solution. The goal, as in any automated ML solution, is to try to offload most of the redundant and time-consuming, repetitive work to the machine while humans can do higher-level cognitive tasks. In the following diagram, you can see the parts of the ML life cycle that SageMaker Autopilot covers:

Figure 6.25 – Lifecycle of Amazon SageMaker

As part of the SageMaker ecosystem, SageMaker Autopilot is tasked with being the automated ML engine. A typical automated ML user flow is defined in the following figure, where a user analyzes the tabular data, selects the target prediction column, and then lets Autopilot do its magic of finding the correct algorithm. The secret sauce here is the underlying Bayesian optimizer as defined by Das et al. in their paper Amazon SageMaker Autopilot: a white box AutoML solution at scale (https://www.amazon...

AWS JumpStart

In Dec 2020, Amazon announced SageMaker JumpStart as a capability to access pre-built model repositories also called model zoos to accelerate model development. Integrated as apart of Amazon SageMaker, JumpStart provides pre-built templates for predictive maintenance, computer vision, autonomous driving, fraud detection, credit risk prediction, OCR for extracting and analyze data from documents, churn prediction, and personalized recommendations.

JumpStart provides an excellent starting point for developers to use these pre-existing templates to JumpStart (pun intended) their development. These accelerator and starter kits are available on GitHub here. https://github.com/awslabs/ and provide recipes and best practices to use Amazon SageMaker model development and deployment mechanisms.

Further details on using AWS JumpStart can be found here. https://docs.aws.amazon.com/sagemaker/latest/dg/studio-jumpstart.html

Summary

In this chapter, you learned about the AWS ML stack and how to get started with AWS SageMaker and notebook development. You also became acquainted with SageMaker Autopilot and its automated ML workflow capabilities. We provided you with an overview of the built-in algorithms, the SageMaker ML life cycle, and what algorithms and techniques are used by SageMaker automated ML. This introduction gives you the background knowledge needed for further exploration and learning of the AWS ML stack and the SageMaker automated ML life cycle.

In the next chapter, we will use some of the SageMaker Autopilot features practically to run classification, regression, and time series analysis.

Further reading

For more information on the following topics, you can refer to the given links:

• Amazon SageMaker Autopilot: a white box AutoML solution at scale by Piali Das et al.: https://www.amazon.science/publications/amazon-sagemaker-autopilot-a-white-box-automl-solution-at-scale
• Build, train, and deploy a ML model with Amazon SageMaker: https://aws.amazon.com/getting-started/hands-on/build-train-deploy-machine-learning-model-sagemaker/
• Amazon SageMaker Studio - AutoML with Amazon SageMaker AutoPilot (part 1): https://www.youtube.com/watch?v=qMEtqJPhqpA
• Amazon SageMaker Studio - AutoML with Amazon SageMaker AutoPilot (part 2): https://www.youtube.com/watch?v=qMEtqJPhqpA&ab_channel=JulienSimon
• SageMaker Studio - AutoML with Amazon SageMaker AutoPilot (part 3): https://www.youtube.com/watch?v=KZSTsWrDGXs&ab_channel=JulienSimon
• SageMaker Studio - AutoML with Amazon SageMaker AutoPilot (part 4): https://www.youtube.com/watch?v=vRHyX3kDstI&ab_channel...