Working with an LLM

There are many AI providers out there that let you call an LLM. In this book, we will use GitHub Models, as that’s a free option, and all you need to use it is a GitHub account. To use GitHub Models, you will either need to start your project in GitHub Codespaces or set up a personal access token (PAT) with the right permissions. The reason you need a token in the first place is that you are calling an API, and the token is used as a bearer token to authenticate the request. To use a local AI model via, for example, Ollama, you wouldn’t need a token. You can type the token directly in the source code, but it’s recommended to keep it in an environment variable for security reasons.

So, what do we need to know if we’ve never worked with AI before? Well, the idea is to send in a prompt and get back a response. The prompt is natural language text that describes what you want the LLM to do. The response is also natural language text that...