Support for Prompt Caching?

[kharbat]

Hello, thanks for the nice work. I'm looking to replace my from-scratch architecture and give this a try - I was wondering if it supports prompt caching

[ilvalerione]

Hi @kharbat, what do you mean with prompt caching? What's the use case you are thinking of?

[kharbat]

Basically to tell the AI that certain prompts contain repetative content/instructions.

https://platform.openai.com/docs/guides/prompt-caching

[ilvalerione]

I'll take a look on that. It's a similar discussion as #285 maybe we can come up with a good and consistent design together.

[jeancodogno]

Hi @kharbat
Just to add to the discussion — prompt caching is already supported when using the OpenAIResponses provider.

You can enable it by passing the prompt_cache_key inside the parameters array, for example:

return new OpenAIResponses(
    key: $apiKey,
    model: $model,
    parameters: [
        'prompt_cache_key' => 'YourPromptCacheKey'
    ]
);

This allows the Responses API to cache repeated parts of your prompt according to OpenAI’s prompt caching guidelines.

One thing that is still missing, though, is registering the cached_tokens field inside the chat_history.
Having that information available would help track cache usage and see when cached content was actually reused.

@ilvalerione — if this is something you’d like to have in the provider, I’d be happy to contribute and open a PR implementing it.

[ilvalerione]

Any improvement suggestions are welcome, but it must implemented at the framework level not on the single provider. You must guarantee the unified messaging layer continue to work across all providers.