The paradigm shift: What is an NLP task?

ChatGPT stunned the world when it suddenly became mainstream in late 2022 and early 2023. An AI could generate human-like text on practically any topic. Thousands of tasks were submitted to this incredible Generative AI transformer. ChatGPT Plus with GPT-4 seemed to be able to perform any task an end user came up with.

However, OpenAI couldn’t have possibly pretrained ChatGPT on thousands of tasks that could not be guessed beforehand. Nor could OpenAI have possibly fine-tuned its GPT models for everything the end user was coming up with.

Of course, a transformer model can be trained for specific tasks and determined downstream tasks such as summarizing. However, models such as ChatGPT can perform downstream tasks for which they were not trained.

This section takes us inside the head of a transformer model to see how the architecture described in Chapter 2, Getting Started with the Architecture of the Transformer Model, applies...