Understanding Text with Natural Language Processing

Natural Language Processing (NLP) is the field of machine learning that deals with the understanding of language, in the form of text data. It is one of the fields that has seen a strong evolution in the last few years, achieving great results in the areas of sentiment analysis, chatbots, text summarization, and machine translation. NLP is at the core of the assistants developed by Amazon (Alexa), Google, and Apple (Siri), as well as modern assistants such as ChatGPT and Llama 2.

In this chapter, we will learn how to use GluonNLP, an MXNet Gluon library specific to NLP, how to build our own networks, and how to use its Model Zoo API for several applications of pre-trained models.

Specifically, we will cover the following topics:

• Introducing NLP networks
• Classifying news highlights with topic modeling
• Analyzing sentiment in movie reviews
• Translating text from Vietnamese to English

Technical requirements

Apart from the technical requirements specified in the Preface, the following technical requirements apply to this chapter:

• Ensure that you have completed Recipe 1, Installing MXNet, Gluon, GluonCV and GluonNLP, from Chapter 1, Up and Running with MXNet
• Ensure that you have completed Recipe 4, Toy dataset for text classification: Load, manage, and visualize a spam email dataset, from Chapter 2, Working with MXNet and Visualizing Datasets: Gluon and DataLoader

The code for this chapter can be found at the following GitHub URL: https://github.com/PacktPublishing/Deep-Learning-with-MXNet-Cookbook/tree/main/ch06

Furthermore, you can access each recipe directly from Google Colab; for example, use the following link for the first recipe of this chapter: https://colab.research.google.com/github/PacktPublishing/Deep-Learning-with-MXNet-Cookbook/blob/main/ch06/6_1_Introducing_NLP_Networks.ipynb.

Introducing NLP networks

In the previous chapters, we saw how different architectures, such as Multi-Layer Perceptrons (MLPs) and Convolutional Neural Networks (CNNs), deal with numerical data and images, respectively. In this recipe, we will analyze the most important architectures to process natural language expressed as text data.

The most important characteristic of natural language is that it is a list of words of variable length, and the order of those words matters; it is a sequence. The previous architectures that we have analyzed are not suited for variable-length data inputs and also do not exploit the relationships among words effectively.

In this recipe, we will introduce neural networks that have been developed to process sequences of words:

1. We will start by applying the network introduced in the previous chapter, that is, CNNs for text processing, called TextCNNs.
2. Afterward, we will introduce Recurrent Neural Networks (RNNs) and their vanilla implementation...

Classifying news highlights with topic modeling

In this recipe, we are going to study one of the most interesting tasks in NLP, topic modeling. In this task, the user must find the number of topics given a set of documents. Sometimes, the topics (and the number of topics) are known beforehand and the supervised learning techniques that we have seen in previous chapters can be applied. However, in a typical scenario, topic modeling datasets do not provide ground truth and are therefore unsupervised learning problems.

To achieve this, we will use a pre-trained model from GluonNLP Model Zoo and apply its word embeddings to feed a clustering algorithm, which will yield the clustered topics. We will apply this process to a new dataset: 1 Million News Headlines.

Getting ready

As in previous chapters, in this recipe, we will be using a little bit of matrix operations and linear algebra, but it will not be hard at all.

Furthermore, we will be working with text datasets. Therefore...

Analyzing sentiment in movie reviews

Sentiment analysis is the use of several different techniques, including NLP, to identify the emotional state associated with human-generated information, text in our case. In this recipe, we are going to perform sentiment analysis on real-world movie reviews. We will classify the reviews into two sentiments: positive or negative.

To achieve this, we will use several pre-trained models from GluonNLP Model Zoo, and apply its word embeddings to feed a classifier, which will output the predicted sentiment. We will apply this process to a new dataset: IMDb Movie Reviews.

Getting ready

As in previous chapters, in this recipe, we will be using a little bit of matrix operations and linear algebra, but it will not be hard at all.

Furthermore, we will be classifying text datasets. Therefore, we will revisit some concepts already seen in Recipe 4, Understanding text datasets – loading, managing, and visualizing the Enron Email dataset,...

Translating text from Vietnamese to English

Translating text automatically (machine translation) has been a very interesting and useful use case for NLP since its inception, as breaking language barriers has lots of applications, including chatbots and automated subtitles in multiple languages.

Before deep learning, machine translation was typically approached as a statistical problem. Even after deep learning, it was not until Google, in 2016, applied deep learning to machine translation that the area of Neural Machine Translation (NMT) was born. This model set the foundation for translating tasks now available in LLMs, such as OpenAI GPT and Google Bard.

In this recipe, we will apply these techniques to translate sentences from Vietnamese to English, using pre-trained models from GluonNLP Model Zoo.

Getting ready

As in previous chapters, in this recipe, we will be using a little bit of matrix operations and linear algebra, but it will not be hard at all.

Furthermore...