Chapter 14: Rapid Prototyping with PyTorch

In the preceding chapters, we have seen multiple facets of PyTorch as a Python library. We have seen its use for training vision and text models. We have learned about its extensive application programming interfaces (APIs) for loading and processing datasets. We have explored the model inference support provided by PyTorch. We have also noticed the interoperability of PyTorch across programming languages (such as C++) as well as with other deep learning libraries (such as TensorFlow).

To accommodate all of these features, PyTorch provides a rich and extensive family of APIs, which makes it one of the best deep learning libraries of all time. However, the vast expanse of those features also makes PyTorch a heavy library, and this can sometimes intimidate users about performing streamlined or simple model training and testing tasks.

This chapter is focused on introducing some of the libraries that are built on top of PyTorch and that...

Technical requirements

We will be using Jupyter Notebooks for all of our exercises. Here is a list of the Python libraries that will be installed for this chapter, using pip (for example, by running pip install torch==1.4.0 on the command line:

jupyter==1.0.0
torch==1.4.0
torchvision==0.5.0 matplotlib==3.1.2
pytorch-lightning==1.0.5
fast.ai==2.1.8

All code files relevant to this chapter are available at the following GitHub page: https://github.com/PacktPublishing/Mastering-PyTorch/tree/master/Chapter14.

Using fast.ai to set up model training in a few minutes

In this section, we will use the fast.ai library (https://docs.fast.ai/) to train and evaluate a handwritten digit classification model in fewer than 10 lines of code, in the form of an exercise. We will also use fast.ai's interpretability module to understand where the trained model is still failing to perform well. The full code for the exercise can be found at the following GitHub page: https://github.com/PacktPublishing/Mastering-PyTorch/blob/master/Chapter14/fast.ai.ipynb.

Setting up fast.ai and loading data

In this section, we will first import the fast.ai library, load the MNIST dataset, and finally preprocess the dataset for model training. We'll proceed as follows:

1. First, we will import fast.ai in the recommended way, as shown here:

   import os
   from fast.ai.vision.all import *

   Although import * is not the recommended way of importing libraries in Python, the fast.ai documentation suggests this format...

Training models on any hardware using PyTorch Lightning

PyTorch Lightning (https://github.com/PyTorchLightning/pytorch-lightning) is yet another library that is built on top of PyTorch to abstract out the boilerplate code needed for model training and evaluation. A special feature of this library is that any model training code written using PyTorch Lightning can be run without changes on any hardware configuration such as multiple CPUs, multiple GPUs, or even multiple TPUs.

In the following exercise, we will train and evaluate a handwritten digit classification model using PyTorch Lightning on CPUs. You can use the same code for training on GPUs or TPUs. The full code for the following exercise can be found here: https://github.com/PacktPublishing/Mastering-PyTorch/blob/master/Chapter14/pytorch_lightning.ipynb.

Defining the model components in PyTorch Lightning

In this part of the exercise, we will demonstrate how to initialize the model class in PyTorch Lightning. This...

Summary

In this final chapter of the book, we focused on both abstracting out the noisy details involved in model training code and the core components to facilitate the rapid prototyping of models. As PyTorch code can often be cluttered with a lot of such noisy detailed code components, we looked at some of the high-level libraries that are built on top of PyTorch.

First, we explored fast.ai, which enables PyTorch models to be trained in fewer than 10 lines of code. In the form of an exercise, we demonstrated the effectiveness of training a handwritten digit classification model using fast.ai. We used one of fast.ai's modules to load the dataset, another module to train and evaluate a model, and—finally—another module to interpret the trained model behavior.

Next, we looked at PyTorch Lightning, which is another high-level library built on top of PyTorch. We did a similar exercise of training a handwritten digit classifier. We demonstrated the code layout...