Skip to content

feat(codex): serve authenticated remote model catalogs - #1124

Open
joshavant wants to merge 5 commits into
mainfrom
codex/remote-codex-catalog-main
Open

joshavant wants to merge 5 commits into
mainfrom
codex/remote-codex-catalog-main

Conversation

@joshavant

@joshavant joshavant commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Custom Codex providers can omit current model metadata and access programs when using a generated local catalog. Add opt-in authenticated GET /v1/codex/models discovery and codex-connect --catalog-mode remote, allowing Codex to refresh complete versioned descriptors through the account and routing policy eligible for inference.

Preserve upstream instructions, unknown metadata and access programs while narrowing models/service tiers to authorized Responses offers. Keep caller/credential/policy caches private and bounded. Keep the catalog route outside the native inference artifact. Add pinned Codex 0.169 coverage. Repair a LiteLLM fixture ordering issue reproduced on unchanged main, preserving every assertion. Include a source deployment hold while qualifying the combined release.

Verification

  • Worker TypeScript; 12,432 Worker tests passed, one skipped. Script suite: 2,763 passed, 102 optional/platform skips. Admin TypeScript, 410 tests and build passed. All provider manifests validate; compiled snapshot matches.
  • All 94 diagnostic checks passed. Deployment-hold tests: two passed. Exact candidate schedule/manual/train checks confirm refusal or no-op before credential access.
  • Native runtime is byte-identical to pinned main; differs from production. Source inspection found no migration or rollback-unsafe marker. Determining control digest is unchanged by the hold.
  • Independent reviews cover feature, Codex fixture/CI changes, test ordering repair and deployment hold; no actionable P0-P2 findings. Reviewers did not independently run tests or reproduce artifacts. An initially unavailable review was retried successfully with the same authorized credential.
  • On current head 6a51eead, all 11 CI jobs, all 46 consumer workflow jobs and CodeQL passed.
  • Hosted ClawSweeper found no implementation defects but requested compatibility/behavior evidence. Add four actual-workerd/Codex 0.169 cases covering fresh CLI/Desktop connections and upgrades from the real pinned v0.5.0 connection writer (608050e4). Released receipts verify without rewriting, remain local through update, and remove/reconnect explicitly enables remote mode. Native clients fetch the actual Worker endpoint, receive filtered model/tier metadata, and complete synthetic inference. Desktop model-list exposes the current display name and Daybreak programs. Auth and unrelated configuration survive update/remove. All four new cases and the complete 82-test native client group passed locally and in hosted CI, with zero skips/failures. CI runs these with the pinned release source. These are synthetic isolated behavior/upgrade results, not production or entitled Blue inference proof. Hosted re-review remains required.

Deployment status

Ready for review; not merged, not release-qualified, not deployed. Candidate 6a51eeadb08be94ae1a62701733623e2bfc6ee88; base 4a434ef28516c74d3ca0452431f6ce27b0dfc294. Production-based fallback preserved separately.

The included hold:true makes scheduled production rollout a no-op and blocks manual/train selection after source landing. Hosted qualification requires reviewed main plus successful exact-source main-push CI. Source landing is authorized after CI and hosted ClawSweeper review are clear; production activation remains separately authorized.

Authorized rehearsal setup: created a dedicated temporary Cloudflare token (expires November 18, 23:00 UTC) and a native-qualification GitHub environment restricted to main, with dedicated Cloudflare/account/admin secrets. Existing credentials and production resources unchanged; no rehearsal compute/storage resources provisioned. A bounded GET probe confirms account/token consistency, subscription access and live capacity. Metered billable usage remains 403 despite Billing Read; costs unknown.

Remaining risk

Main includes native changes beyond production; the catalog itself remains edge-only. Four fresh isolated synthetic cutover/recovery fixtures, actual saved-session/accounting/recovery evidence, cleanup verification and a reviewed registry entry remain required. The native qualification registry is empty. Temporary credentials must be removed after rehearsal/expiry. No production rollout is authorized by this PR.

Remote discovery establishes metadata visibility, not Daybreak Blue inference. The prior live Blue request returned upstream 403 access_program_not_enabled. Retest after authorized deployment, retaining the assumption that the deployed key has Blue access.

@clawsweeper

clawsweeper Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🦞👀
ClawSweeper picked this up.

Pull request received. I will update this pull request when review starts.

ClawSweeper review complete

ClawSweeper finished reviewing this revision. The review result is being finalized.

View the workflow run.

@joshavant
joshavant marked this pull request as ready for review November 17, 2026 22:35
@clawsweeper clawsweeper Bot added clawsweeper:intake:feature Preliminary submission classification; full source review pending. P2 Normal priority bug or improvement with limited blast radius. merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. and removed clawsweeper:intake:feature Preliminary submission classification; full source review pending. labels Nov 17, 2026
@clawsweeper

clawsweeper Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Codex review: needs maintainer review before merge.

What this changes

Adds authenticated remote Codex model catalogs and an opt-in connection mode that preserves existing local connections.

Example: Connect Codex 0.169.0 to gpt-6-sol with --catalog-mode remote.

  • Before: Codex reads generated local model metadata.
  • After: Codex refreshes authorized descriptors through /v1/codex/models while inference continues through the native Responses route.

Review scores

Measure Result What it means
Overall readiness 🧂 unranked krab (1/6) The patch appears correct and compatibility coverage is materially improved; synthetic fixture evidence does not clear the required real-behavior proof gate.
Proof confidence 🧂 unranked krab (1/6) Real behavior proof is not required for maintainer- or bot-authored pull requests.
Patch quality 🐚 platinum hermit (4/6) No actionable review findings were identified.

Product

Kind: Feature · Worth it: Yes · Fix scope: Complete
User problem: Generated local catalogs can omit current upstream model metadata and access-program visibility.
Reason: The maintainer-authored direction provides current descriptors through an explicit opt-in while preserving existing local connections and inference authorization.

Merge readiness

✅ Ready for maintainer review

Keep open: this maintainer-authored PR adds a useful capability absent from main, and inspection found no concrete implementation defect.

Priority: P2
Reviewed head: 6a51eeadb08be94ae1a62701733623e2bfc6ee88

Before merge

None.

Findings

None.

Tests

  • Missing end-to-end proof: Provide redacted terminal output or a recording of the installed connector and Codex client retrieving current metadata through the Worker with a real qualified upstream account; paid inference and Daybreak entitlement are not required.
Agent review details

How this fits together

The Worker authenticates catalog requests, resolves existing inference eligibility and credentials, fetches upstream descriptors, and returns privately cached metadata filtered to authorized models and tiers.

flowchart LR
  A[Codex client] --> B[Worker catalog endpoint]
  B --> C[Existing authorization and offers]
  C --> D[Selected OpenAI credential]
  D --> E[Upstream model descriptors]
  E --> F[Shared catalog projection]
  F --> A
Loading

Technical review

Best possible solution:

Retain the opt-in endpoint and existing connection owners, preserve released local connections, and keep production activation separately gated.

Do we have a high-confidence way to reproduce the issue?

This is a new capability; current main demonstrably uses local catalogs, and the supplied fixtures exercise the proposed remote path.

Is this the best way to solve the issue?

The implementation reuses existing authorization, credential selection, catalog projection, and configuration ownership rather than introducing competing decision paths.

AGENTS.md: found and applied where relevant.

Codex review notes: model internal, reasoning medium; reviewed against e2727c39ddda.

Provenance checked

  • Catalog projection and descriptor preservation keeps the original intent (f837292: Preserve complete official descriptors while selecting reachable routes and narrowing paid tiers; later operation-offer qualification made eligibility key-scoped.)
  • Connection receipts and restoration keeps the original intent (feat(codex): manage Desktop root connections #203: Restore routing settings atomically while preserving authentication, preferences, comments, and unrelated configuration.)
  • Production deployment selection changes intended behavior with a stated reason (ce9705a: Resume guarded deployment after recovery protections landed; this PR expressly reinstates the hold for separate combined-release qualification.)
  • LiteLLM retirement assertion ordering keeps the original intent (feat(litellm): declare scoped native model discovery #806: Require original producer cancellation and joins before claiming retirement; the reordered reads retain the assertions and join before inspecting retirement state.)

Testing

Proof path: in-process harness.

Security

None.

Evidence

What I checked:

  • Authorization and credential boundary: The endpoint authenticates before resolving eligible native Responses offers, selects credentials through upstreamAuth, requires one qualified account, and partitions cached metadata by caller, policy, credential revision, and client version. (worker/codex-models.ts:95, 6a51eeadb08b)
  • Released connection compatibility: The new upgrade cases execute the actual v0.5.0 connection writer, verify its unmodified receipts, preserve local mode through update, and check authentication and unrelated configuration after removal for CLI and Desktop. (test/codex-remote-upgrade.test.mjs:91, 6a51eeadb08b)
  • Hosted validation and its inspection limit: GitHub reports successful execution of the native-client step at the exact reviewed head: https://github.com/openclaw/clawrouter/actions/runs/44197488961/job/134442617719. The source wires in the released writer and remote client. Raw log retrieval redirected to a sandbox-blocked artifact host, so the reported 82-test diagnostics were not independently read. (.github/workflows/ci.yml:97, 6a51eeadb08b)
  • Current main retains local catalog setup: Fetched main still installs model_catalog_json and lacks the new remote endpoint and shared projection module; the central capability is not already implemented. (scripts/codex-connect.mjs:49, e2727c39ddda)
  • Accepted deployment hold: The maintainer-authored PR explicitly authorizes source landing while deliberately stopping scheduled and manual production selection until separate qualification and activation; that accepted tradeoff requires no second direction decision. (config/deployment-hold.json:3, 6a51eeadb08b)
  • Original connection ownership contract: feat(codex): manage Desktop root connections #203 established atomic routing restoration and preservation of authentication and unrelated settings; the remote mode retains the existing receipt and TOML-editing owners. (scripts/codex-connect.mjs:177, c36d1b6cb171)

Preflight source

Source unknown. Source comparison at the observed PR snapshot only. Completion is self-reported; body, policy, tests, execution, and report bytes are not verified.

Labels

Label changes:

  • add status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Label justifications:

  • P2: Useful opt-in metadata discovery with limited impact on existing connections.
  • merge-risk: 🚨 automation: The source deployment hold intentionally blocks production deployment selection after landing; the maintainer-authored PR explicitly accepts this operational effect.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🧂 unranked krab and patch quality is 🐚 platinum hermit.
  • status: 👀 ready for maintainer look: ClawSweeper has no concrete contributor-facing blocker left for this PR.

Rating scale

6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior.

Workflow

ClawSweeper edits this one comment on every review. Comment @clawsweeper re-review for a fresh review only. Repair and merge execution lanes are retired.

History

Review history (1 earlier review cycle)
  • reviewed 2026-11-17T22:51:48.976Z sha 1e304ce :: blocked before merge. :: none

Reviewed November 17, 2026, 6:57 PM ET / 23:57 UTC (Revision 2).

@joshavant

Copy link
Copy Markdown
Contributor Author

@clawsweeper re-review

Please review current head 6a51eeadb08be94ae1a62701733623e2bfc6ee88, including the compatibility and behavior proof requested in the previous review.

test/codex-remote-upgrade.test.mjs runs the official Codex 0.169 client against the current Worker in workerd/Miniflare, using real KV/DO/admission paths and a deterministic synthetic upstream. Four hosted cases pass: fresh CLI, CLI upgrade from v0.5.0, fresh Desktop, and Desktop upgrade from v0.5.0. Upgrade inputs are generated by executing the actual released connection writer and catalog helper at 608050e4e4dae5ebc93919e0759ffb4bf0664528; receipts are not reconstructed or edited.

The tests verify that released receipts load without rewriting, updates preserve local mode, remove/reconnect explicitly enables remote mode, native clients fetch the actual Worker catalog endpoint, unauthorized models and unpriced tiers are filtered, Daybreak descriptors survive, synthetic inference succeeds, and authentication/unrelated configuration survive update and removal. Desktop model/list also verifies current metadata and daybreakBlue visibility.

Hosted native proof: all 82 tests passed, zero failures or skips. The four cases emit explicit evidence diagnostics in that log.

All current-head checks have passed: CI, consumer contract, and CodeQL.

These are isolated actual-runtime/native-client results with a synthetic upstream, not production or entitled Daybreak Blue inference proof. The source deployment hold remains enabled; production activation is outside this PR's authorization.

@clawsweeper

clawsweeper Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

🦞👀
Exact review queued.

Re-review progress:

@clawsweeper clawsweeper Bot added the status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. label Nov 17, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-risk: 🚨 automation 🚨 Merging this PR could break CI, automerge, proof capture, label sync, or automation. P2 Normal priority bug or improvement with limited blast radius. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant