[Upstream] [codex] fix channel cache refresh after channel changes#118
Open
Sagit-chu wants to merge 1 commit into
Open
[Upstream] [codex] fix channel cache refresh after channel changes#118Sagit-chu wants to merge 1 commit into
Sagit-chu wants to merge 1 commit into
Conversation
- refresh in-memory channel selection state after channel create, update, delete, and status changes - invalidate cached per-group model lists so model discovery and routing stay consistent - add regression coverage for channel cache refresh behavior - document the cache refresh behavior in changelog and FAQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Synced from upstream PR: songquanpeng#2372
Summary
/v1/modelsstays consistent with request routingRoot Cause
On Redis-backed deployments, startup enables the in-memory channel selection cache and refreshes it on the periodic sync loop. Channel mutations updated the database and
abilitiestable immediately, but request routing could still use stale in-memory channel state until the next sync. During that window, users could see newly available models from/v1/modelswhile actual requests failed withno available channels.Testing
env -u GOROOT -u GOPATH -u GOTOOLDIR CGO_ENABLED=1 /opt/homebrew/bin/go test ./model ./controller/...git diff --check