Chapter 2: Getting Started with PyTorch 1.x for NLP

PyTorch is a Python-based machine learning library. It consists of two main features: its ability to efficiently perform tensor operations with hardware acceleration (using GPUs) and its ability to build deep neural networks. PyTorch also uses dynamic computational graphs instead of static ones, which sets it apart from similar libraries such as TensorFlow. By demonstrating how language can be represented using tensors and how neural networks can be used to learn from NLP, we will show that both these features are particularly useful for natural language processing.

In this chapter, we will show you how to get PyTorch up and running on your computer, as well as demonstrate some of its key functionalities. We will then compare PyTorch to some other deep learning frameworks, before exploring some of the NLP functionality of PyTorch, such as its ability to perform tensor operations, and finally demonstrate how to build a simple neural...

Technical requirements

For this chapter, Python needs to be installed. It is recommended to use the latest version of Python (3.6 or higher). It is also recommended to use the Anaconda package manager to install PyTorch. A CUDA-compatible GPU is required to run tensor operations on a GPU. All the code for this chapter can be found at https://github.com/PacktPublishing/Hands-On-Natural-Language-Processing-with-PyTorch-1.x.

Installing and using PyTorch 1.x

Like most Python packages, PyTorch is very simple to install. There are two main ways of doing so. The first is to simply install it using pip in the command line. Simply type the following command:

pip install torch torchvision

While this installation method is quick, it is recommended to install using Anaconda instead, as this includes all the required dependencies and binaries for PyTorch to run. Furthermore, Anaconda will be required later to enable training models on a GPU using CUDA. PyTorch can be installed through Anaconda by entering the following in the command line:

conda install torch torchvision -c pytorch

To check that PyTorch is working correctly, we can open a Jupyter Notebook and run a few simple commands:

1. To define a Tensor in PyTorch, we can do the following:

   import torch
   x = torch.tensor([1.,2.])
   print(x)

   This results in the following output:

   Figure 2.1 – Tensor output

   This shows that tensors within PyTorch...

Enabling PyTorch acceleration using CUDA

One of the main benefits of PyTorch is its ability to enable acceleration through the use of a graphics processing unit (GPU). Deep learning is a computational task that is easily parallelizable, meaning that the calculations can be broken down into smaller tasks and calculated across many smaller processors. This means that instead of needing to execute the task on a single CPU, it is more efficient to perform the calculation on a GPU.

GPUs were originally created to efficiently render graphics, but since deep learning has grown in popularity, GPUs have been frequently used for their ability to perform multiple calculations simultaneously. While a traditional CPU may consist of around four or eight cores, a GPU consists of hundreds of smaller cores. Because calculations can be executed across all these cores simultaneously, GPUs can rapidly reduce the time taken to perform deep learning tasks.

Consider a single pass within a neural network...

Comparing PyTorch to other deep learning frameworks

PyTorch is one of the main frameworks used in deep learning today. There are other widely used frameworks available too, such as TensorFlow, Theano, and Caffe. While these are very similar in many ways, there are some key differences in how they operate. These include the following:

• How the models are computed
• The way in which the computational graphs are compiled
• The ability to create dynamic computational graphs with variable layers
• Differences in syntax

Arguably, the main difference between PyTorch and other frameworks is in the way that the models themselves are computed. PyTorch uses an automatic differentiation method called autograd, which allows computational graphs to be defined and executed dynamically. This is in contrast to other frameworks such as TensorFlow, which is a static framework. In these static frameworks, computational graphs must be defined and compiled before finally being executed...

Building a simple neural network in PyTorch

We will now walk through building a neural network from scratch in PyTorch. Here, we have a small .csv file containing several examples of images from the MNIST dataset. The MNIST dataset consists of a collection of hand-drawn digits between 0 and 9 that we want to attempt to classify. The following is an example from the MNIST dataset, consisting of a hand-drawn digit 1:

Figure 2.11 – Sample image from the MNIST dataset

These images are 28x28 in size: 784 pixels in total. Our dataset in train.csv consists of 1,000 of these images, with each consisting of 784 pixel values, as well as the correct classification of the digit (in this case, 1).

Loading the data

We will begin by loading the data, as follows:

1. First, we need to load our training dataset, as follows:

   train = pd.read_csv("train.csv")
   train_labels = train['label'].values
   train = train.drop("label",axis=1...

NLP for PyTorch

Now that we have learned how to build neural networks, we will see how it is possible to build models for NLP using PyTorch. In this example, we will create a basic bag-of-words classifier in order to classify the language of a given sentence.

Setting up the classifier

For this example, we'll take a selection of sentences in Spanish and English:

1. First, we split each sentence into a list of words and take the language of each sentence as a label. We take a section of sentences to train our model on and keep a small section to one side as our test set. We do this so that we can evaluate the performance of our model after it has been trained:

   ("This is my favourite chapter".lower().split(),\
    "English"),
   ("Estoy en la biblioteca".lower().split(), "Spanish")

   Note that we also transform each word into lowercase, which stops words being double counted in our bag-of-words. If we have the word book and the word Book...

Summary

In this chapter, we introduced PyTorch and some of its key features. Hopefully, you now have a better understanding of how PyTorch differs from other deep learning frameworks and how it can be used to build basic neural networks. While these simple examples are just the tip of the iceberg, we have illustrated that PyTorch is an immensely powerful tool for NLP analysis and learning.

In future chapters, we will demonstrate how the unique properties of PyTorch can be utilized to build highly sophisticated models for solving very complex machine learning tasks.