Creating a RAG system

In the previous sections, we explored various approaches to interact with Foundation Models – more precisely, available LLMs from Kaggle Models. First, we experimented with prompting, directly using the models. Then, we quantized the models with two different approaches. We also showed that we can use models to generate code. A more complex application included a combination of LangChain with an LLM to create sequences of connected operations, or task sequences.

In all these cases, the answers of the LLM are based on the information already available with the model at the time of training it. If we would like to have the LLM answer queries about information that was never presented to the LLM, the model might provide a deceiving answer by hallucinating. To counter this tendency of models to hallucinate when they do not have the right information, we can fine-tune models with our own data. The disadvantage to this is that it is costly, since the computational...