LLM embeddings as an alternative to fine-tuning
ChatGPT models are impressive. They have taken everyone by surprise. However, ChatGPT has a memory problem! It only remembers what it learned from its training data. For example, in January 2024, ChatGPT’s cutoff date was April 2023. It cannot answer questions about events after April 2023. OpenAI has found a workaround for some issues using the BING search engine, but this isn’t enough.
Also, ChatGPT only knows what the training set contains. For example, maybe you have information that hasn’t been made public and that ChatGPT cannot find.
In this chapter, we will build two methods:
- An ask method using Retrieval Augmented Generation (RAG) by adding information to the prompt
- A RAG search and ask function that leverages the Ada embedding model
In both cases, these approaches take us from prompt design to advanced prompt engineering.
