Multiple Function Calls

[mosnicholas]

What is the best way to handle a chat request that has multiple function calls and requires something of a for-loop? OpenAI's plugins show a great example of being able to accomplish this.

https://platform.openai.com/docs/guides/gpt/function-calling

For example, if you ask the model “Find the weather in Boston this weekend, book dinner for two on Saturday, and update my calendar” and provide the corresponding functions for these queries, it may choose to call them back to back and only at the end create a user-facing message.

Screen.Recording.2023-08-16.at.1.10.48.PM.mov

[mosnicholas]

@MaxLeiter quick thought -- maybe a better function signature for experimental_onFunctionCall would be:

experimental_onFunctionCall?: (functionCallPayload: FunctionCallPayload, createFunctionCallMessages: (functionCallResult: JSONValue) => CreateMessage[]) => ReadableStream;

The reason for this change would be to return an OpenAI stream from the experimental_onFunctionCall method, which would allow us to make the calls actually be recursive. The example that comes to mind is if you have the following function:

  {
    name: 'get_current_weather',
    description: 'Get the current weather',
    parameters: {
      type: 'object',
      properties: {
        location: {
          type: 'string',
          description: 'The city and state, e.g. San Francisco, CA'
        },
        format: {
          type: 'string',
          enum: ['celsius', 'fahrenheit'],
          description:
            'The temperature unit to use. Infer this from the users location.'
        }
      },
      required: ['location', 'format']
    }
  }
]

and call the chat with the following request: What's the weather in SF and in NYC?, we would have to call the function twice sequentially. In the current format, we can't actually accomplish that, we can only call it in one loop (ie. get first function call, call requested function & synthesize response).

[MaxLeiter]

Have you seen https://sdk.vercel.ai/docs/guides/providers/openai-functions#recursive-function-calls? You can pass functions to your nested openai calls

[MaxLeiter]

Side note: I need to update that section for openai v4

[mosnicholas]

I actually tested this an hour ago and realized this is a non-problem. I didn’t realize the original stream keeps executing after its returned data already, this is excellent.

Next question: to contain costs and prevent it from recursing forever, is there a maxIter count we could pass in?

[MaxLeiter]

you can keep one yourself pretty easily but no, we don't currently track the depth. We'd like to in the future but we're still adjusting the API.