Adding custom tools to assistants

OpenAI, Ollama, Google Gemini, and Anthropic chats have, when instructed how, the capability to request function calls to be performed. Assistants mirror that capability.

To use it, we first need to define the function.

tool := GtLlmFunctionTool new
		name: 'lookupDocumentationPage';
		parameters: {'pageName'};
		description: 'Looks up a documentation page by name and returns its content in Markdown format. `pageName` may be a substring. If multiple pages are found, only the first one will be returned.';
		block: [ :aFunctionCall | 
			(LeDatabase gtBook asyncPagesForWord: aFunctionCall anyArgument) toArray wait
				ifEmpty: [ 'No page found' ]
				ifNotEmpty: [ :pages | pages first asMarkdownPage ] ]
  

We can then add it to the provider. Please note that due to a technical limitation, tools and structured outputs do not play well together with Ollama assistants [1], Anthropic does not formally support structured outputs [2], and Google Gemini does not support both at the same time [3]. Otherwise, both providers are equivalent.

First we create an assistant.

assistant := GtLlmAssistant new
		description: 'You are an assistant that answers questions about Glamorous Toolkit (also: GToolkit or GT) by referring to the GToolkit book through tool search.'.

assistant
  

Finally, we can use the chat as before, adding the tool.

chat := assistant createChat.
chat provider addTool: tool.
chat sendChatRequest: 'Tell me about gt4llm.'
  

[1] This is because OpenAI uses a signal to interrupt the conversation to request a tool call and then resumes answering the question. In Ollama, the tool call is embedded in the answer, thus it’s an in-band communication channel.

[2] It is however usually able to adhere to the format even when not forced to.

[3] When both are set, we set the tools in the API and rely on the system prompt to enforce the format. It doesn’t work reliably, but seems to perform pretty well, similarly to Anthropic.