Access to chat memory in LlmRanker

[smilyanovr]

Hi!

I've been looking into Autonomy usage which internally will use the LlmRanker in order to decide which the best goal is based in user's input.

Let's assume the following scenario:

1. User ask a question
2. A goal is selected by the LlmRanker
3. Selected goal might end up with asking user for more context/information
4. Then user provides answer
5. Next call by LlmRanker will try to choose the next best goal only by the user input provided in 4) without information on what happened so far
6. Next goal selection might become a bit unreliable

I imagine that LlmRanker needs to have access to the previous messages (chat memory) in order to compute the next goal in the best possible way.

What do you think about this?

[johnsonr]

In Embabel, chat memory is not included by default. There is no conversation history unless the user chooses to use the Conversation interface, which is automatically used by a chatbot.

In the case of a chatbot, Autonomy is less necessary because you can have multiple actions to respond to UserMessage. See the guide and ragbot repositories.

I'd like to understand more about your use case. I'm converting this to a discussion for now; we can raise an issue if we figure out a necessary action.

[smilyanovr]

Thanks for the feedback! I think that the chatbot will do the trick I need. I missed this part in Embabel.

I want to share couple of things which I noticed today.

I wanted to provide my own Chat agent so I can pass a few more things like additional exclusion types. Then I noticed AchievableGoalsToolGroupFactory might skip some excluded types because the check for JvmType.isAssignableFrom ended up having the internal clazz field initialized with one class loader, while the comparable class was from another class loader. Still not sure how this happened, but I will try to find simple reproducible steps and help with a PR.

Second thing is that it might be a good idea to somehow pass the parent process id from GoalToolCallback when the goal agent is executed. This information will be valuable if the selected goal involves several actions with LLM interactions. In this way we can produce additional responses to the end user. What do you think on this one?

Third thing is that somehow we might need to pass bindings from the chat agent process to the children processes which are launched by the GoalToolCallback. I have this use case where some internal business objects need to be added into the blackboard and passed down to the children processes. I guess this whole thing can be achieved as adding a new ToolCallback into the hierarchy which does all this (including point 2).