PyTorch Fundamentals

In the previous chapter, we learned about the fundamental building blocks of a neural network and also implemented forward and back-propagation from scratch in Python.

In this chapter, we will dive into the foundations of building a neural network using PyTorch, which we will leverage multiple times in subsequent chapters when we learn about various use cases in image analysis. We will start by learning about the core data type that PyTorch works on – tensor objects. We will then dive deep into the various operations that can be performed on tensor objects and how to leverage them when building a neural network model on top of a toy dataset (so that we strengthen our understanding before we gradually look at more realistic datasets, starting with the next chapter). This will allow us so to gain an intuition of how to build neural network models using...

Installing PyTorch

PyTorch provides multiple functionalities that aid in building a neural network – abstracting the various components using high-level methods and also providing us with tensor objects that leverage GPUs to train a neural network faster.

Before installing PyTorch, we first need to install Python, as follows:

1. To install Python, we'll use the anaconda.com/distribution/ platform to fetch an installer that will install Python as well as important deep learning-specific libraries for us automatically:

Choose the graphical installer of the latest Python version 3.xx (3.7, as of the time of writing this book) and let it download.

2. Install it using the downloaded installer:

Choose the Add Anaconda to my PATH environment variable option during installation as this will make it easy to invoke Anaconda's version of Python when we type python in Command Prompt/Terminal.

Next, we'll install PyTorch, which is equally simple.

3. Visit the QUICK START LOCALLY...

PyTorch tensors

Tensors are the fundamental data types of PyTorch. A tensor is a multi-dimensional matrix similar to NumPy's ndarrays:

• A scalar can be represented as a zero-dimensional tensor.
• A vector can be represented as a one-dimensional tensor.
• A two-dimensional matrix can be represented as a two-dimensional tensor.
• A multi-dimensional matrix can be represented as a multi-dimensional tensor.

Pictorially, the tensors look as follows:

For instance, we can consider a color image as a three-dimensional tensor of pixel values, since a color image consists of height x width x 3 pixels – where the three channels correspond to the RGB channels. Similarly, a grayscale image can be considered a two-dimensional tensor as it consists of height x width pixels.

By the end of this section, we will learn why tensors are useful and how to initialize them, as well as perform various operations on top of tensors. This will serve as a base for when we study leveraging tensors to build...

Building a neural network using PyTorch

In the previous chapter, we learned about building a neural network from scratch, where the components of a neural network are as follows:

• The number of hidden layers
• The number of units in a hidden layer
• Activation functions performed at the various layers
• The loss function that we try to optimize for
• The learning rate associated with the neural network
• The batch size of data leveraged to build the neural network
• The number of epochs of forward and back-propagation

However, for all of these, we built them from scratch using NumPy arrays in Python. In this section, we will learn about implementing all of these using PyTorch on a toy dataset. Note that we will leverage our learning so far regarding initializing tensor objects, performing various operations on top of them, and calculating the gradient values to update weights when building a neural network using PyTorch.

Note that, in this chapter, to gain the intuition of performing various operations...

Using a sequential method to build a neural network

So far, we have built a neural network by defining a class where we define the layers and how the layers are connected with each other. In this section, we will learn about a simplified way of defining the neural network architecture using the Sequential class. We will perform the same steps as we have done in the previous sections, except that the class that was used to define the neural network architecture manually will be substituted with a Sequential class for creating a neural network architecture.

Let's code up the network for the same toy data that we have worked on in this chapter:

The following code is available as Sequential_method_to_build_a_neural_network.ipynb in the Chapter02 folder of this book's GitHub repository - https://tinyurl.com/mcvp-packt

1. Define the toy dataset:

x = [[1,2],[3,4],[5,6],[7,8]]
y = [[3],[7],[11],[15]]

2. Import the relevant packages and define the device we will work on:

import torch...

Saving and loading a PyTorch model

One of the important aspects of working on neural network models is to save and load back a model after training. Think of a scenario where you have to make inferences from an already-trained model. You would load the trained model instead of training it again.

The following code is available as save_and_load_pytorch_model.ipynb in the Chapter02 folder of this book's GitHub repository - https://tinyurl.com/mcvp-packt

Before going through the relevant commands to do that, taking the preceding example as our case, let's understand what all the important components that completely define a neural network are. We need the following:

• A unique name (key) for each tensor (parameter)
• The logic to connect every tensor in the network with one or the other
• The values (weight/bias values) of each tensor

While the first point is taken care of during the __init__ phase of a definition, the second point is taken care of during the forward method definition...

Summary

In this chapter, we learned about the building blocks of PyTorch – tensor objects and performing various operations on top of them. We proceeded further by building a neural network on a toy dataset where we started by building a class that initializes the feed-forward architecture, fetching data points from the dataset by specifying the batch size, and defining the loss function and the optimizer, looping through multiple epochs. Finally, we also learned about defining custom loss functions to optimize a metric of choice and leveraging the sequential method to simplify the process of defining the network architecture.

All the preceding steps form the foundation of building a neural network, which will be leveraged multiple times in the various use cases that we will build in subsequent chapters.

With this knowledge of the various components of building a neural network using PyTorch, we will proceed to the next chapter, where we will learn about the various practical...

Questions

1. Why should we convert integer inputs into float values during training?
2. What are the various methods to reshape a tensor object?
3. Why is computation faster with tensor objects over NumPy arrays?
4. What constitutes the init magic function in a neural network class?
5. Why do we perform zero gradients before performing back-propagation?
6. What magic functions constitute the dataset class?
7. How do we make predictions on new data points?
8. How do we fetch the intermediate layer values of a neural network?
9. How does the sequential method help in simplifying defining the architecture of a neural network?