Add prompt cache key as a parameter when starting Codex #5949
Conversation
aibrahim-oai
changed the title
|
@codex review this |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review this |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| auth_manager, | ||
| history, | ||
| self.session_source.clone(), | ||
| Some(conversation_id.to_string()), | ||
| ) |
There was a problem hiding this comment.
Passing Some(conversation_id.to_string()) here only works for the very first fork of a base conversation, where the prompt cache key equals the conversation id. Once you fork a fork (or any conversation whose session was started with a custom prompt cache key), the current conversation's cache key is inherited from its ancestor via Session::get_prompt_cache_key(), not from its own id. By hard-coding the new fork to use its immediate conversation id as the key, we break cache affinity for nested forks and lose the optimization this change is trying to introduce. We should look up the existing conversation's prompt cache key and propagate that through Codex::spawn instead of defaulting to the conversation id.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
That's true. I need to think about it more tomorrow.
This will help us keep cache when delegating to Codex using the same history. i.e. in compact tasks
It also helps us associating forked task to the same prompt cache key
The change basically adds
prompt_cache_keyas an optional parameter onCodex::Spawn. Then, we pass it toSession::new. We add the parameter as a member ofSessionstruct (source of truth). Then, we add it onclientto propagate it to the API.clientis constructed every turn fromSession