Alternative for Semantic Kernel IChatHistoryReducer?

[jooooel]

Is there (or are there plans for) an equivalent to the IChatHistoryReducer that exist in Semantic Kernel?

https://learn.microsoft.com/en-us/semantic-kernel/concepts/ai-services/chat-completion/chat-history?pivots=programming-language-csharp

[stephentoub]

IChatReducer and friends
https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.ai.ichatreducer?view=net-9.0-pp
https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.ai.reducingchatclientbuilderextensions.usechatreducer?view=net-9.0-pp

The latter middleware applies at the IChatClient level and so can be plugged into a ChatClientAgent. We may want to introduce equivalent middleware using the same interface at the AIAgent level.

cc: @rogerbarreto, @crickman, @MackinnonBuck, @westey-m

[westey-m]

Yes, this is supported via the factory. This is so that users of the agent don't have to know the underlying agent type and agent thread when interacting with an AIAgent.

AIAgent agent = new AzureOpenAIClient(
    new Uri(endpoint),
    new AzureCliCredential())
     .GetChatClient(deploymentName)
     .CreateAIAgent(new ChatClientAgentOptions
     {
         Name = JokerName,
         Instructions = JokerInstructions,
         ChatMessageStoreFactory = ctx => new InMemoryChatMessageStore(new MessageCountingChatReducer(1), ctx.SerializedState, ctx.JsonSerializerOptions)
     });

[stephentoub]

This is so that users of the agent don't have to know the underlying agent type and agent thread when interacting with an AIAgent.

@westey-m, if I specify InMemoryChatMessageStore, what happens if the underlying agent is Assistants? What would it look like to enable ChatClientAgentThread to be used in a similar manner, so that the reducer would be used iff messages were stored on the client?

[westey-m]

If using assistants (or any other service that requires service storage), the provided chat message store does not get used at all, regardless of the type of store provided. It is not great that the store only gets used in some circumstances, but we of course don't know if the storage is service based or not until the first run.
For service based storage, the service is responsible for keeping the chat history manageable. Assistants for example, states that it truncates threads automatically, so there is no AF support for managing that.

[stephentoub]

I get all that.

Let me try again. Your example shows using InMemoryChatMessageStore with ChatClientAgent. Won't that fail here?

agent-framework/dotnet/src/Microsoft.Agents.AI/ChatClient/ChatClientAgent.cs

Line 593 in 6285419

chatOptions.ConversationId = typedThread.ConversationId;

once it gets back a thread ID from the underlying IChatClient if it's based on assistants and tries to set the conversation ID once the store has already been set? How do I make this work where the store I provide is used if messages are being stored client side but ignored if the service is handling it?

[westey-m]

How do I make this work where the store I provide is used if messages are being stored client side but ignored if the service is handling it?

Right, looks like we have a regression here. We either need to allow switching ChatClientAgentThread to a different mode (store vs id), or not set the mode until we know what the right type is. I think we had both approaches at some stage, but I think the latter makes sense since it stops us from accidentally switching mode, which really never works.

I've created an issue and will send a PR for it asap: #1228