Unleash the Power of Generative AI with Kaggle Models

In the previous chapters, our primary focus was on mastering the analysis of diverse data types and developing strategies to tackle a variety of problems. We delved into an array of tools and methodologies for data exploration and visualization, enriching our skill set in these areas. A few of the earlier chapters were dedicated to constructing baseline models, notably for participation in competitive scenarios.

Now, in this current chapter, we will pivot our attention toward leveraging Kaggle Models. Our objective is to integrate these models into Kaggle applications, in order to prototype the use of the newest Generative AI technologies in practical applications. A few examples of such real-world applications are personalized marketing, chatbots, content creation, targeted advertising, answering customers’ inquiries, fraud detection, medical diagnosis, patient monitoring, drug discovery, personalized medicine, financial...

Introducing Kaggle Models

Kaggle Models represent one of the latest innovations on the Kaggle platform. This feature gained prominence in particular after the introduction of code competitions, where participants often train models either on their local hardware or in the cloud. Post-training, they upload these models to Kaggle as a dataset. This practice allows Kagglers to utilize these pre-trained models in their inference notebooks, streamlining the process for code competition submissions. This method significantly reduces the runtime of the inference notebooks, fitting within the stringent time and memory constraints of the competition. Kaggle’s endorsement of this approach aligns well with real-world production systems, where model training and inference typically occur in separate pipelines.

This strategy becomes indispensable with large-scale models, such as those based on Transformer architectures, considering the immense computational resources required for fine...

Prompting a foundation model

LLMs can be used directly, for example, for such tasks as summarization, question answering, and reasoning. Due to the very large amounts of data on which they were trained, they can answer very well to a variety of questions on many subjects, since they have the context available in that training dataset.

In many practical cases, such LLMs can correctly answer our questions on the first attempt. In other cases, we will need to provide a few clarifications or examples. The quality of the answers in these zero-shot or few-shot approaches highly depends on the ability of the user to craft the prompts for LLM. In this section, we will show the simplest way to interact with one LLM on Kaggle, using prompts.

Model evaluation and testing

Before starting to use an LLM on Kaggle, we will need to perform a few preparation steps. We begin by loading the model and then defining a tokenizer. Next, we create a model pipeline. In our first code example,...

Building a multi-task application with Langchain

Langchain is the most popular task-chaining framework (Reference 8). Task chaining is an extension of the prompt engineering concept that we illustrated in the previous section. Chains serve as predetermined sequences of operations, designed to organize intricate processes in a format that is both more manageable and easier to understand. These chains follow a distinct order of actions. They are well suited for workflows characterized by a consistent number of steps. With task chaining, you can create sequences of prompts, where the output of the previous task executed by the framework is fed as input for the next task.

Besides Langchain, there are now several other options available for task chaining, like LlamaIndex or Semantic Kernel (from Microsoft). Langchain provides multiple functionalities, including specialized tools to ingest data or output results, intelligent agents, as well as the possibility to extend it by defining...

Code generation with Kaggle Models

For code generation, we will experiment with the Code Llama model, the 13b version. From the LLMs available on the Kaggle platform at the time of writing, this model was the most appropriate, in regards to its purpose (it is a model specialized for code generation) and size (i.e., we can use it with Kaggle Notebooks), for the task of code generation. The notebook used to demonstrate the code generation is given in Reference 9. The model is loaded, quantized using bitsandbytes, and has a tokenizer initialized in the same way, as demonstrated in Reference 7. We define a prompt and a pipeline (using the transformers function) with the following code:

prompt = 'Write the code for a function to compute the area of circle.'
sequences = pipeline(
    prompt,
    do_sample=True,
    top_k=10,
    temperature=0.1,
    top_p=0.95,
    num_return_sequences=1,
    eos_token_id=tokenizer.eos_token_id,
    max_length=200,
)

The result of executing...

Creating a RAG system

In the previous sections, we explored various approaches to interact with Foundation Models – more precisely, available LLMs from Kaggle Models. First, we experimented with prompting, directly using the models. Then, we quantized the models with two different approaches. We also showed that we can use models to generate code. A more complex application included a combination of LangChain with an LLM to create sequences of connected operations, or task sequences.

In all these cases, the answers of the LLM are based on the information already available with the model at the time of training it. If we would like to have the LLM answer queries about information that was never presented to the LLM, the model might provide a deceiving answer by hallucinating. To counter this tendency of models to hallucinate when they do not have the right information, we can fine-tune models with our own data. The disadvantage to this is that it is costly, since the computational...

Summary

In this chapter, we explored how we can leverage the potential of Generative AI, using LLMs from Kaggle Models. We started by focusing on the simplest way to use such Foundation Models – by directly prompting them. We learned that crafting a prompt is important and experimented with simple math questions. We used the models that were available in Kaggle Models as well as quantized ones and quantized models with two approaches: using Llama.cpp and the bitsandbytes library. We then combined Langchain with a LLM to create sequences of chained tasks, where the output of one task is used to craft (by the framework) the input (or prompt) for the next task. Using the Code Llama 2 model, we tested the feasibility of code generation on Kaggle. The results were less than perfect, with multiple sequences generated besides the expected one. Finally, we learned how to create a RAG system that combines the speed, versatility, and ease of using vector databases with the chaining functions...

References

1. Llama 2, Kaggle Models: https://www.kaggle.com/models/metaresearch/llama-2
2. Mistral, Kaggle Models: https://www.kaggle.com/models/mistral-ai/mistral/
3. Gabriel Preda – Test Llama v2 with math, Kaggle Notebooks: https://www.kaggle.com/code/gpreda/test-llama-v2-with-math
4. Model Quantization, HuggingFace: https://huggingface.co/docs/optimum/concept_guides/quantization
5. Gabriel Preda – Test Llama 2 quantized with Llama.cpp, Kaggle Notebooks: https://www.kaggle.com/code/gpreda/test-llama-2-quantized-with-llama-cpp
6. Gabriel Preda – Test of Llama 2 quantized with llama.cpp (on CPU), Kaggle Notebooks: https://www.kaggle.com/code/gpreda/test-of-llama-2-quantized-with-llama-cpp-on-cpu
7. Gabriel Preda – Simple sequential chain with Llama 2 and Langchain, Kaggle Notebooks: https://www.kaggle.com/code/gpreda/simple-sequential-chain-with-llama-2-and-langchain/
8. Langchain, Wikipedia page: https://en.wikipedia.org/wiki...