Fundamentals of generative AI

The fundamental of GenAI always revolves around FMs. These FMs are pre-trained on vast amounts of unstructured data and contain a large number of parameters, sometimes in the billions, which makes the FMs capable of learning new complex concepts. FMs that are used for natural language processing, such as the ones from OpenAI’s GPT-3 and GPT-4, which are used in Chat-GPT, are pre-trained on a diverse range of internet text, enabling them to learn patterns, grammar, and general knowledge from vast amounts. These FMs are also called large language models (LLMs).

FMs differ from other ML models in several ways:

• Scale: FMs are trained on massive amounts of data, often involving billions of parameters. This large scale allows them to capture complex patterns and relationships in the data.
• Pre-training and fine-tuning: FMs undergo a two-step training process. First, they are pre-trained on a large corpus of publicly available text from the...