gt4llm

gt4llm is a framework for working with Large Language Models (LLMs) from within Glamorous Toolkit.

It allows users to chat with LLMs as well as manage their models, fine-tune, build custom prompts and assistants, and more.

For an overview of the system, read this section of the book. For case studies, refer to the Building an LLM assistant for editing blog posts and

Building an LLM assistant for creating Wardley Maps
case studies.

Working with OpenAI

The most advanced integration is with OpenAI. An example can be found in the case studies or in the gt4openai: working with OpenAI section of this book.

Before you work with it, please set up the API key first: Adding an OpenAI API key

Working with Ollama

It's possible to work with Ollama as well. For example:

Working with the Ollama API client talks about the basic API access.

Using structured outputs and Using modelfiles talks about more high level concepts.

Working with Anthropic

Finally, there is an Anthropic integration. It is the simplest of the LLM integrations and can serve as a case study of how to add new LLM integrations. Nonetheless, it supports all the chat use cases including tool calls.