Text generation and the magic of LSTMs

In the previous sections, we discussed different ways of representing textual data in order to make it fit for consumption by different NLP algorithms. In this section, we will leverage this understanding of text representation to work our way toward building text generation models.

So far, we have built models using feedforward networks consisting of different kinds and combinations of layers. These networks work with one training example at a time, which is independent of other training samples. We say that the samples are independent and identically distributed, or IID. Language, or text, is a bit different.

As we discussed in the previous sections, words change their meaning based on the context they are being used in. In other words, if we were to develop and train a language generation model, we would have to ensure the model understands the context of its input.

Recurrent Neural Networks (RNNs) are a class of neural networks...