Manual Invoke Support

[tmnguyen1403]

In Semantic Kernel (SK), I have the option to manual invoke the functions/tools. This allows me to filter out sensitive information before sending it to LLM model like username/password and only replace them when the LLM model response with the tool call and the filtered arguments. I am looking for the equivalent in AgentFramework but not being able to look for it so far. Is the capability implemented or any plan to support it?

Sample code from SK:

execution_settings = AzureChatPromptExecutionSettings()
execution_settings.function_choice_behavior = FunctionChoiceBehavior.Auto(auto_invoke=False)
result = await chat_completion_service.get_chat_message_content(chat_history=history,
settings=execution_settings, kernel=kernel)

[moonbox3]

Hi @tmnguyen1403, thanks for your question. We don't have the equivalent of auto-invoke yet with our current ToolMode, which allows you to specify the type of function calling.

In the meantime, I would have a look at our middleware, which is a similar concept to SK's filters, but I think more powerful. You can perform this filtering before sending a tool response to the model. We have samples here, please give it a read: https://github.com/microsoft/agent-framework/blob/main/python/samples/getting_started/middleware/README.md

[stephentoub]

The .NET implementation of AIFunction has it deriving from AIFunctionDeclaration, with the derived AIFunction just adding the InvokeAsync method. If an AIFunctionDeclaration that's not AIFunction is put into the tools collection, the FunctionInvokingChatClient will allow it to pass through back to the caller without being invoked, since it can't be invoked. Thus, for any function you want to allow to be auto-invoked, you can just put them into the Tools and FunctionInokingChatClient will automatically invoke calls to them, but for any you want to handle manually, you can put in an AIFunctionDeclaration-only wrapper, such as one created with function.AsDeclarationOnly(). You can also similarly translate an AIFunction into one that requires the caller to approve its invocation; by wrapping the AIFunction with an ApprovalRequiredAIFunction, when such a tool has a call made to it, the FunctionInvokingChatClient will translate it into an approval requested content that's issued back to the caller, and then only when the caller sends back in an approval response content will the FunctionInvokingChatClient be permitted to invoke it.

#1131 is already adding the approval support for Python. We should aim to add the AIFunctionDeclaration-equivalent as well.

[eavanvalkenburg]

I just created a PR that adds support for declaration only functions, so between that and using approval_mode you can control what actually happens with your functions!