Help me understand ChatMessageStoreFactory - is there a bug?

[toresenneseth]

This question is about .NET.

This might be a bug in agent framework, or more likely a bug sitting between my chair and my keyboard...

I don't understand how the ChatMessageStoreFactory / ChatMessageStore is supposed to work if the intention is to enable rehydrating a chat history.
When an agent runs, the ChatMessageStore.GetMessagesAsync is not called on the first request, meaning no chat history is available.
GetMessagesAsync is however called on all subsequent requests.

The only way I can persist the chat history is by storing the AgentThread to a database, and then load it back up the next time the agent runs. (My agent - and application - shuts down between runs).

Is this by design? Because ChatMessageStoreFactory is not async, I does not seem like I'm suppose to initialize the ChatMessageStore manually, right?

[toresenneseth]

Something like this works as a "hack" to make sure the ChatMessageStore is initialized before the first call to RunAsync or RunStreamingAsync. I can deserialize an empty thread which will create a new instance of a ClientAgentThread internally, where the ChatMessageStore is initialized and assigned to the thread.

ChatClientAgent agent = new ChatCompletionsClient(new Uri("my-url"),
    new Azure.AzureKeyCredential("my-key"))
    .AsIChatClient("gpt-4o")
    .CreateAIAgent(new ChatClientAgentOptions
    {
        Instructions = instructions,
        ChatMessageStoreFactory = (context) => new SqlServerChatMessageStore(chatSessionId, connectionString, context),
        ChatOptions = new ChatOptions
        {
            Tools = [AIFunctionFactory.Create(MyTools.CalculateTax), AIFunctionFactory.Create(MyTools.SetEmployerTax)],
        }
    });

// Hack to force creation of the ChatMessageStore before the first call to RunAsync
AgentThread agentThread = agent.DeserializeThread(JsonSerializer.Deserialize<JsonElement>("{}"));

[moonbox3]

@westey-m is this something you could help with?

[toresenneseth]

Note that this works fine, and there are probably good reasons why it's made like this, but the API might not be intuitive. Other people seem to be a little confused as well: #2145

[westey-m]

@toresenneseth, the expectation is that the message store implementation would generate an id to store the chat history under and then store that id with the Agent Thread state (if needed). The AgentThread therefore always has a reference to the id. This id should be serialized and deserialized with the AgentThread when the AgentThread is stored between runs.

I think we have a gap where you already have a database of chat history that you want to use, but don't have any stored agent threads. We are looking at adding more support in this area at the moment, so expect some changes.

Also see the following sections for some more context:

• https://learn.microsoft.com/en-us/agent-framework/user-guide/agents/agent-memory?pivots=programming-language-csharp#3rd-party-chat-history-storage
• https://learn.microsoft.com/en-us/agent-framework/tutorials/agents/third-party-chat-history-storage?pivots=programming-language-csharp#serialization
• https://learn.microsoft.com/en-us/agent-framework/tutorials/agents/third-party-chat-history-storage?pivots=programming-language-csharp#sample-chatmessagestore-implementation

I've created a work item to track this and provide a solution: #2325
Please take a look and feel free to comment and add thoughts.

[toresenneseth]

Thank you for the clarification - and yes, my thinking was that why would I need to store the thread separately when I have a chat message store db. Will take a look at the work item you created, thank you!