Exploring state-of-the-art NLP machine learning

While the techniques we have learned in this book so far are highly useful methodologies for training our own machine learning model from scratch, they are far from the most sophisticated models being developed globally. Companies and research groups are constantly striving to create the most advanced machine learning models that will achieve the highest performance on a number of NLP tasks.

Currently, there are two NLP models that have the best performance and could be considered state-of-the-art: BERT and GPT-2. Both models are forms of generalized language models. We will discuss these in more detail in the upcoming sections.

BERT

BERT, which stands for Bidirectional Encoder Representations from Transformers, was developed by Google in 2018 and is widely considered to be the leading model in the field of NLP, having achieved leading performance in natural language inference and question-answering tasks. Fortunately, this has...