In the last chapter, we talked about document classification, and a special case of document classification called sentiment classification. In doing so, we got to talk quite a bit about vectorization.
In this chapter, we're going to keep talking about solving NLP problems, but instead of classifying, we're going to generate new sequences of words.
We will cover the following topics in this chapter:
The networks that we've looked at so far have done some truly amazing things. But they've all had one pretty big limitation: they can only be applied to problems where the output is of a fixed and well-known size.
Sequence-to-sequence models are able to map sequences of inputs to sequences of outputs with variable lengths.
When using a sequence-to-sequence model, we will take a sequence in and get a sequence out in exchange. These sequences don't have to be the same length. Sequence-to-sequence models allow us to learn a mapping between the input sequence and the output sequence.
There are a variety of applications where sequence-to-sequence models might be useful, and we will talk about those applications next.
...Je ne parle pas français. That's how you say I don't speak French in English. Just about two years ago, I found myself in Paris, speaking almost no French. I had read a book and listened to some DVDs before I went, but even after a few months of practice, my mastery of the French language was pretty much pathetic. Then, on the very first morning of my trip, I woke up and walked into a nearby boulangerie (a French or French-style bakery) for my breakfast and morning coffee. I did my best at Bonjour, parlez-vous anglais? They didn't speak a bit of English, or perhaps they were enjoying my struggle. Either way, when my breakfast depended on my mastery of French, I was more motivated to struggle through Je voudrais un pain au chocolat (translation: I would like one of those delicious chocolate bread things) than I had ever been. I was quickly...
In this chapter, we covered the basics of sequence-to-sequence models, including how they work and how we can use them. Hopefully, we've shown you a powerful tool for machine translation, question-answering, and chat applications.
If you've made it this far, good job. You've seen quite a few applications of deep learning and you're finding yourself on the right of the bell curve toward the state-of-the-art in the application of deep neural networks.
In the next chapter, I'm going to show you an example of another advanced topic, deep reinforcement learning, or deep-Q learning, and show you how to implement your own deep-Q network.
Until then, sois détendu!
© 2018 Packt Publishing Limited All Rights Reserved
