gt4llm: Working with LLMs
gt4llm is a framework for working with Large Language Models (LLMs) from within Glamorous Toolkit.
It allows users to chat with LLMs as well as manage their models, fine-tune, build custom prompts and assistants, and more.
The most advanced integration is with OpenAI. An example can be found in the case studies or in the Working with OpenAI section of this book.
Before you work with it, please set up the API key first: Adding an OpenAI API key
It's possible to work with Ollama as well. For example:
Working with the Ollama API client talks about the basic API access.
Using structured outputs and Using modelfiles talks about more high level concepts.
Note that the models used in the examples are chosen because they are small and thus more likely to run on any given system. They're not chosen because the provide good results.
There is also an Anthropic integration. It is the simplest of the LLM integrations and can serve as a case study of how to add new LLM integrations. Nonetheless, it supports all the chat use cases including tool calls and vision.
To use it, you will have to add an API key.
Finally, there is an integration for Google Gemini. Much like Anthropic, it is simple, but because it deviates a bit from the format of the other providers, it is a bit more involved, and might serve as a case study for adding providers that do things a bit differently.
It, too, Adding a Gemini API key.