Sending images to LLMs

Some LLM providers have vision capabilities. As such, chats can work with images through the UI or programmatically.

To attach an image through the UI, simply attach them using the Attach icon at the bottom of the message editor.

The same can be accomplished programmatically. In the following, we illustrate attaching a screenshot of GT programmatically for illustration purposes.

First, we create a chat using the default provider:

chat := GtLChat new
  

We then create a screenshot of the current window of Glamorous Toolkit.

fileReference := BlExporter png
		element: (GtWorldElement allInstances detect: [:aWorldElement | aWorldElement space isFocused]);
		export
  

Finally, we can send this screenshot to the chat.

chat sendFiles: {fileReference} withMessage: 'What do you see?'