You're reading from The Machine Learning Solutions Architect Handbook

Product type Book

Published in Jan 2022

Publisher Packt

ISBN-13 9781801072168

Pages 442 pages

Edition 1st Edition

Languages

Python

Concepts

Machine Learning

Author (1):

David Ping

Table of Contents (17) Chapters

Preface

1. Section 1: Solving Business Challenges with Machine Learning Solution Architecture

2. Chapter 1: Machine Learning and Machine Learning Solutions Architecture

3. Chapter 2: Business Use Cases for Machine Learning

4. Section 2: The Science, Tools, and Infrastructure Platform for Machine Learning

5. Chapter 3: Machine Learning Algorithms

6. Chapter 4: Data Management for Machine Learning

7. Chapter 5: Open Source Machine Learning Libraries

8. Chapter 6: Kubernetes Container Orchestration Infrastructure Management

9. Section 3: Technical Architecture Design and Regulatory Considerations for Enterprise ML Platforms

10. Chapter 7: Open Source Machine Learning Platforms

11. Chapter 8: Building a Data Science Environment Using AWS ML Services

12. Chapter 9: Building an Enterprise ML Architecture with AWS ML Services

13. Chapter 10: Advanced ML Engineering

14. Chapter 11: ML Governance, Bias, Explainability, and Privacy

15. Chapter 12: Building ML Solutions with AWS AI Services

16. Other Books You May Enjoy

Hands-on lab – running distributed model training with PyTorch

In this hands-on lab, you will use SageMaker Training Service to run data parallel distributed training. We will use PyTorch's torch.nn.parallel.DistributedDataParallel API as the distributed training framework and run the training job on a small cluster. We will reuse the dataset and training scripts from the hands-on lab in Chapter 8, Building a Data Science Environment Using AWS Services.

All right, let's get started!

Modifying the training script

First, we need to add distributed training support to the training script. To start, create a copy of the train.py file, rename the file train-dis.py, and open the train-dis.py file. You will need to make changes to the following three main functions. The following steps are meant to highlight the key changes needed. To run the lab, you can download the modified train-dis.py file from https://github.com/PacktPublishing/The-Machine-Learning-Solutions...