Building an LLM assistant for editing blog posts
By default, the gt4llm framework comes with an assistant for editing blog posts. To get a background on what an assistant is, refer to Handling custom instructions using assistants. For the purposes of this case study, it can be conceptualized as an AI agent with custom instructions.
This particular example relies on OpenAI. Before using it, please go to Adding an OpenAI API key to ensure you have the key setup.
As example, let's consider we have a string with a post we want to write. Like this one:
There are at least two ways to use LLMs to support the understanding of software systems: (A) use LLMs to generate summaries (B) use LLMs to generate tools with which you generate summaries The difference might appear subtle, but it's significant. A number of recent articles propose mostly (A) as a direction. These use LLMs as solution generators. However, any concrete solutions entail biases. When we get the final answer, these biases are invisible. The solution might be correct, but you'd have no real way of knowing what tradeoffs it contains. The alternative is to regard current LLMs as generators of possibilities that should be evaluated against reality before being relied upon for decision-making in a technical space. That is why I favor (B), especially when combined with a moldable development environment. When we get the tools, we have a chance to review the bias and apply other forms of checks as well. We can then use these tools to produce visualizations of the system ourselves. LLMs work interestingly well for shallow questions (I call shallow the questions whose answers can be evaluated against reality quickly). Moldable Development splits technical decision-making into small, shallow problems addressed by small, shallow tools. As such, it can be a fertile target for LLMs.
blogPostText := thisSnippet previousSibling text
And now, let's say we want to have a chat about it with a dedicated LLM:
GtOpenAIBlogPostAssistant new createChatOn: blogPostText
The above assistant has been evolved from a generic assistant. We can refine a generic assistant to get to a similar result. This page demonstrates how to do that.
We start by instantiating a generic assistant with a description.
assistant := GtLlmAssistant new description: 'You are an assistant that is used to interactively work on blog posts.'
We want the assistant to work with a Post format, so we create it and give information on how to serialize and deserialize the blog post for the assistant.
assistant addFormat: (GtLlmAssistantFormatDescription new type: 'string'; format: 'Text'; name: 'Post'; priority: 10). serializableText := GtLlmSerializableValueHolder new name: 'Post'; content: blogPostText; serializer: #asString; updater: [ :aText :aString | aText deleteAll appendString: aString; yourself ]
From there, we can create a chat and tell it to work with our text.
chat := assistant createChat. chat onInstance: serializableText
As of now, the assistant has no specific actions, however. Let’s add a simple one now.
assistant addAction: (GtLlmAssistantAction new name: 'Create title'; priority: 10; description: 'Create title. Put result in the `Text` property. Do not use quotes of any kind. Be crisp.')
Ideally, we would add examples, but since we have none, a description of the action will need to be enough. From here, we can reify the assistant. To do so, click on the reify button on any of the messages in the chat. You will be allowed to create an assistant class as well as a message class. The latter then allows you to add custom views for your messages, such as the diff view in GtOpenAIBlogPostAssistant
above.