Our world is rich with natural language data. Over the past several decades, the way we communicate with one another has shifted to the digital realm and, as such, this data can be used to build models that can improve our online experience. From returning relevant results within a search engine, to autocompleting the next word you type in an email, the benefits of being able to extract insights from natural language is clear to see.

While the way we, as humans, understand language differs notably from the way a model or artificial intelligence may understand it, by shedding light on machine learning and what it is used for, we can begin to understand just how these deep learning models understand language and what fundamentally happens when a model learns from data.

Throughout this book, we will explore this application of artificial intelligence and deep learning to natural language. Through the use of PyTorch,...

Fundamentally, machine learning is the algorithmic process used to identify patterns and extract trends from data. By training specific machine learning algorithms on data, a machine learning model may learn insights that aren't immediately obvious to the human eye. A medical imaging model may learn to detect cancer from images of the human body, while a sentiment analysis model may learn that a book review containing the words good, excellent, and entertaining is more likely to be a positive review than one containing the words bad, terrible, and boring.

Broadly speaking, machine learning algorithms fall into two main categories: supervised learning and unsupervised learning.

Supervised learning

Supervised learning covers any task where we wish to use an input to predict an output. Let's say we wish to train a model to predict house prices. We know that larger houses tend to sell for more money, but we don't know the exact...

In our previous examples, we have discussed mainly regressions in the form . We have touched on using polynomials to fit more complex equations such as . However, as we add more features to our model, when to use a transformation of the original feature becomes a case of trial and error. Using neural networks, we are able to fit a much more complex function, y = f(X), to our data, without the need to engineer or transform our existing features. 

Structure of neural networks

When we were learning the optimal value of , which minimized loss in our regressions, this is effectively the same as a one-layer neural network:

Figure 1.10 – One-layer neural network

Here, we take each of our features, , as an input, illustrated here by a node. We wish to learn the parameters, , which are represented as connections in this diagram. Our final sum of all the products between and gives us our final prediction, y:

A neural network...

Unlike humans, computers do not understand text – at least not in the same way that we do. In order to create machine learning models that are able to learn from data, we must first learn to represent natural language in a way that computers are able to process.

When we discussed machine learning fundamentals, you may have noticed that loss functions all deal with numerical data so as to be able to minimize loss. Because of this, we wish to represent our text in a numerical format that can form the basis of our input into a neural network. Here, we will cover a couple of basic ways of numerically representing our data. 

Bag-of-words

The first and most simple way of representing text is by using a bag-of-words representation. This method simply counts the words in a given sentence or document and counts all the words. These counts are then transformed into a vector where each element of the vector is the count of the times each word in the...

In this chapter, we introduced the fundamentals of machine learning and neural networks, as well as a brief overview of transforming text for use within these models. In the next chapter, we will provide a brief overview of PyTorch and how it can be used to construct some of these models.