Neural networks are a sub-type of machine learning methods that are inspired by the structure and function of the biological brain, such as the biological neuron shown in Figure 1.2. In neural networks, each computational unit, analogically called a neuron, is connected to other neurons in a layered fashion. When the number of such layers is more than two, the neural network thus formed is called a Deep Neural Network (DNN). Such models are generally called deep learning models.

Figure 1.2: Artificial neuron inspired by biological neuron. (Biological neuron image by: https://pixabay.com/users/clker-free-vector-images-3736)

Deep learning models have been proven superior to other classical machine learning models because of their ability to learn highly complex relationships between input data and the output (ground truth). In recent times, deep learning has gained a lot of attention, and rightly so, primarily because of the following two reasons...

So far, we have spoken of how a neural network structure is built. In order to train a neural network, we need to adopt an optimization schedule. Like any other parameter-based machine learning model, a deep learning model is trained by tuning its parameters. The parameters are tuned through the process of backpropagation, wherein the final or output layer of the neural network yields a loss. This loss is calculated with the help of a loss function that takes in the neural network’s final layer’s outputs and the corresponding ground truth target values. This loss is then backpropagated to the previous layers using gradient descent and the chain rule of differentiation.

The parameters or weights at each layer are accordingly modified in order to minimize the loss. The extent of modification is determined by a coefficient, which varies from 0 to 1, also known as the learning rate. This whole procedure of updating the weights of a neural network...

PyTorch is a machine learning library for Python based on the Torch library. PyTorch is extensively used as a deep learning tool both for research as well as building industrial applications. It is primarily developed by Meta. PyTorch is competition for the other well-known deep learning library – TensorFlow, which is developed by Google. The initial difference between these two was that PyTorch was based on eager execution whereas TensorFlow was built on graph-based deferred execution. Although, TensorFlow now also provides an eager execution mode.

Eager execution is basically an imperative programming mode where mathematical operations are computed immediately. A deferred execution mode would have all the operations stored in a computational graph without immediate calculations and then the entire graph would be evaluated later. Eager execution is considered advantageous for reasons such as intuitive flow, easy debugging...

In this chapter, we refreshed deep learning concepts, explored the PyTorch deep learning library in contrast to TensorFlow and ran a hands-on exercise on training a deep learning model (CNN) from scratch.

In the next chapter, we will take a deeper look at the gamut of different CNN architectures developed over the years, how each of them is uniquely useful, and how they can be easily implemented using PyTorch.

1. The Asimov Institute, The Neural Network Zoo: https://www.asimovinstitute.org/neural-network-zoo/
2. TensorFlow, Module: tf.dtypes: https://www.tensorflow.org/api_docs/python/tf/dtypes
3. PyTorch, CUDA Semantics: https://pytorch.org/docs/stable/notes/cuda.html
4. Nvidia Developer, About CUDA: https://developer.nvidia.com/about-cuda
5. The MNIST Database: http://yann.lecun.com/exdb/mnist/
6. GitHub 1: https://github.com/arj7192/MasteringPyTorchV2/blob/main/Chapter01/mnist_pytorch.ipynb
7. GitHub 2: https://github.com/arj7192/MasteringPyTorchV2/blob/main/Chapter01/mnist_tensorflow.ipynb
8. GitHub 3: https://github.com/pytorch/examples/tree/master/mnist

Learn more on Discord

Join our community’s Discord space for discussions with the author and other readers:

https://packt.link/mastorch