Skip to content

Commit 360d275

Browse files
samxu01claude
andcommitted
chore(litellm): retire env-var codex auth path (ADR-014 Phase A)
Pod-side device-auth via the codex-cli sidecar (ADR-014) is the live auth source for ChatGPT/Codex. The env-var-fed path (laptop-device-auth tokens uploaded to GCP SM → secret → env var) is dead-on-arrival under cluster-IP-bound OAuth — those tokens 401 token_invalidated on first cluster use regardless of JWT exp. - codex-auth-rotator: drop env-var fallback branch in get_candidates; pod-side /chatgpt-auth/auth-N.json is the only source. Drop unused OPENAI_CODEX_*_{1,2,3} env vars (only CLIENT_ID remains for OAuth refresh). - codex-auth-seed init: replace 150-line env-var seeder with a 40-line pod-side seeder so LiteLLM has auth.json ready before boot (avoids startup race with rotator sidecar). Drops all OPENAI_CODEX_* env vars from init container too. - LiteLLM main container: drop unused OPENAI_CODEX_ACCESS_TOKEN[_2|_3] env vars; litellm-config no longer references them. Net: 212 lines of legacy auth code gone. Phase B follow-ups (#371-#373) cover clawdbot env vars, the daily refresh job, and the presets.ts audit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 14d3354 commit 360d275

1 file changed

Lines changed: 46 additions & 212 deletions

File tree

k8s/helm/commonly/templates/agents/litellm-deployment.yaml

Lines changed: 46 additions & 212 deletions
Original file line numberDiff line numberDiff line change
@@ -174,29 +174,11 @@ spec:
174174
name: api-keys
175175
key: anthropic-api-key
176176
optional: true
177-
# chatgpt/ provider reads auth.json from CHATGPT_TOKEN_DIR (written by init container).
178-
# Account-1 token is written to auth.json; accounts 2 & 3 use api_key in litellm_params.
177+
# chatgpt/ provider reads auth.json from CHATGPT_TOKEN_DIR (written by the
178+
# codex-auth-rotator sidecar; source files seeded by operator via the codex-cli
179+
# sidecar). See ADR-014.
179180
- name: CHATGPT_TOKEN_DIR
180181
value: /chatgpt-auth
181-
# Codex account tokens — account-1 goes to auth.json; accounts 2 & 3 use api_key.
182-
- name: OPENAI_CODEX_ACCESS_TOKEN
183-
valueFrom:
184-
secretKeyRef:
185-
name: api-keys
186-
key: openai-codex-access-token
187-
optional: true
188-
- name: OPENAI_CODEX_ACCESS_TOKEN_2
189-
valueFrom:
190-
secretKeyRef:
191-
name: api-keys
192-
key: openai-codex-access-token-2
193-
optional: true
194-
- name: OPENAI_CODEX_ACCESS_TOKEN_3
195-
valueFrom:
196-
secretKeyRef:
197-
name: api-keys
198-
key: openai-codex-access-token-3
199-
optional: true
200182
# Force prompt/response storage in spend logs regardless of runtime general_settings.
201183
# The config-file setting (store_prompts_in_spend_logs: true) is sometimes shadowed by
202184
# the in-memory general_settings dict at startup; the env var is the reliable fallback.
@@ -364,15 +346,10 @@ spec:
364346
def get_candidates():
365347
"""Build the rotation candidate list.
366348
367-
PREFERRED: pod-side device-auth'd files at /chatgpt-auth/auth-{1,2,3}.json.
368-
Operator created these via `kubectl exec` into the codex-cli sidecar +
369-
`codex login --device-auth`. Tokens are cluster-IP-bound, so ChatGPT
370-
doesn't invalidate them on cluster usage (the inverse of the env-var
371-
path below where laptop-bound sessions die on first cluster call).
372-
373-
FALLBACK: env-var-fed tokens from GCP SM. Kept for backward compat with
374-
older operator flows; flagged stale by `--mode envvar` in logs so
375-
it's clear when we're on the dead path.
349+
Pod-side device-auth'd files at /chatgpt-auth/auth-{1,2,3}.json.
350+
Operator created these via `kubectl exec` into the codex-cli sidecar +
351+
`codex login --device-auth`. Tokens are cluster-IP-bound, so ChatGPT
352+
doesn't invalidate them on cluster usage. See ADR-014.
376353
"""
377354
pod_files = [
378355
('1', '/chatgpt-auth/auth-1.json'),
@@ -385,20 +362,6 @@ spec:
385362
if rec:
386363
access, refresh, id_tok = rec
387364
out.append((label, access, refresh, id_tok))
388-
if out:
389-
return out
390-
# Fallback: env-var path (legacy, pre-cluster-bound)
391-
specs = [
392-
('1', 'OPENAI_CODEX_ACCESS_TOKEN', 'OPENAI_CODEX_REFRESH_TOKEN', 'OPENAI_CODEX_ID_TOKEN'),
393-
('2', 'OPENAI_CODEX_ACCESS_TOKEN_2', 'OPENAI_CODEX_REFRESH_TOKEN_2', ''),
394-
('3', 'OPENAI_CODEX_ACCESS_TOKEN_3', 'OPENAI_CODEX_REFRESH_TOKEN_3', 'OPENAI_CODEX_ID_TOKEN_3'),
395-
]
396-
for label, a_env, r_env, i_env in specs:
397-
access = os.environ.get(a_env, '')
398-
refresh = os.environ.get(r_env, '')
399-
id_tok = os.environ.get(i_env, '') if i_env else ''
400-
if access or refresh:
401-
out.append((label, access, refresh, id_tok))
402365
return out
403366
404367
def write_auth(label, access, refresh, id_tok, exp):
@@ -466,30 +429,6 @@ spec:
466429
# Can be overridden per-deploy via Helm if you have one dedicated
467430
# cluster-only account and don't need rotation at all.
468431
value: "1800"
469-
- name: OPENAI_CODEX_ACCESS_TOKEN
470-
valueFrom:
471-
secretKeyRef: {name: api-keys, key: openai-codex-access-token, optional: true}
472-
- name: OPENAI_CODEX_REFRESH_TOKEN
473-
valueFrom:
474-
secretKeyRef: {name: api-keys, key: openai-codex-refresh-token, optional: true}
475-
- name: OPENAI_CODEX_ID_TOKEN
476-
valueFrom:
477-
secretKeyRef: {name: api-keys, key: openai-codex-id-token, optional: true}
478-
- name: OPENAI_CODEX_ACCESS_TOKEN_2
479-
valueFrom:
480-
secretKeyRef: {name: api-keys, key: openai-codex-access-token-2, optional: true}
481-
- name: OPENAI_CODEX_REFRESH_TOKEN_2
482-
valueFrom:
483-
secretKeyRef: {name: api-keys, key: openai-codex-refresh-token-2, optional: true}
484-
- name: OPENAI_CODEX_ACCESS_TOKEN_3
485-
valueFrom:
486-
secretKeyRef: {name: api-keys, key: openai-codex-access-token-3, optional: true}
487-
- name: OPENAI_CODEX_REFRESH_TOKEN_3
488-
valueFrom:
489-
secretKeyRef: {name: api-keys, key: openai-codex-refresh-token-3, optional: true}
490-
- name: OPENAI_CODEX_ID_TOKEN_3
491-
valueFrom:
492-
secretKeyRef: {name: api-keys, key: openai-codex-id-token-3, optional: true}
493432
- name: OPENAI_CODEX_CLIENT_ID
494433
valueFrom:
495434
secretKeyRef: {name: api-keys, key: openai-codex-client-id, optional: true}
@@ -569,21 +508,22 @@ spec:
569508
cpu: 200m
570509
memory: 256Mi
571510
initContainers:
572-
# Write the best available (non-expired) Codex token to auth.json for LiteLLM's chatgpt/ provider.
573-
# Account-1 entries in litellm-config have no api_key — the chatgpt/ provider reads
574-
# CHATGPT_TOKEN_DIR/auth.json at startup. Accounts 2 & 3 use api_key in litellm_params directly.
575-
# Priority: account-3 first (quota available), then account-1 as fallback, else empty (disables chatgpt/ provider).
576-
# NOTE: chatgpt/ provider ignores api_key in litellm_params — all accounts share this single auth.json.
577-
# There is no per-deployment rotation; whichever account is written here is used for ALL chatgpt/ calls.
511+
# Seed /chatgpt-auth/auth.json from a pod-side device-auth'd file before LiteLLM
512+
# starts, so the chatgpt/ provider has a token to read on first boot. This avoids
513+
# a startup race with codex-auth-rotator (which runs as a sidecar and only writes
514+
# auth.json on its first scheduled tick). If no pod-side auth-N.json exists yet
515+
# (fresh cluster — operator hasn't run auth-login.sh), writes empty auth.json
516+
# and the chatgpt/ provider stays disabled until the operator seeds an account.
517+
#
518+
# All OAuth refresh + multi-account rotation logic lives in the rotator sidecar.
519+
# See ADR-014.
578520
- name: codex-auth-seed
579521
image: python:3-alpine
580522
command:
581523
- python3
582524
- -c
583525
- |
584-
import os, sys, base64, json, datetime, urllib.request, urllib.parse, urllib.error
585-
586-
OAUTH_URL = 'https://auth.openai.com/oauth/token'
526+
import json, os, sys, base64
587527
588528
def token_exp(tok):
589529
if not tok:
@@ -595,150 +535,44 @@ spec:
595535
except Exception:
596536
return 0
597537
598-
def refresh_token(refresh_tok, client_id, label):
599-
"""Exchange refresh_token for fresh access_token. Returns (access, refresh, id_token, exp) or None."""
600-
if not refresh_tok or not client_id:
601-
return None
602-
body = urllib.parse.urlencode({
603-
'grant_type': 'refresh_token',
604-
'refresh_token': refresh_tok,
605-
'client_id': client_id,
606-
'scope': 'openid profile email',
607-
}).encode('utf-8')
608-
req = urllib.request.Request(
609-
OAUTH_URL,
610-
data=body,
611-
headers={'Content-Type': 'application/x-www-form-urlencoded'},
612-
)
538+
def read_pod_auth(path):
613539
try:
614-
with urllib.request.urlopen(req, timeout=15) as resp:
615-
payload = json.loads(resp.read().decode('utf-8'))
616-
except urllib.error.HTTPError as e:
617-
err_body = e.read().decode('utf-8', errors='replace')[:200]
618-
print(f' [{label}] refresh failed: HTTP {e.code} {err_body}')
619-
return None
620-
except Exception as e:
621-
print(f' [{label}] refresh failed: {e}')
540+
with open(path) as f:
541+
d = json.load(f)
542+
except Exception:
622543
return None
623-
new_access = payload.get('access_token', '')
624-
new_id = payload.get('id_token', '')
625-
new_refresh = payload.get('refresh_token', refresh_tok)
626-
exp = token_exp(new_access) or (int(datetime.datetime.now(datetime.timezone.utc).timestamp()) + int(payload.get('expires_in', 0) or 0))
627-
if not new_access:
628-
print(f' [{label}] refresh returned no access_token')
544+
tokens = d.get('tokens') if isinstance(d.get('tokens'), dict) else d
545+
access = tokens.get('access_token', '') or d.get('access_token', '')
546+
refresh = tokens.get('refresh_token', '') or d.get('refresh_token', '')
547+
id_tok = tokens.get('id_token', '') or d.get('id_token', '')
548+
if not (access or refresh):
629549
return None
630-
return (new_access, new_refresh, new_id, exp)
631-
632-
now_ts = int(datetime.datetime.now(datetime.timezone.utc).timestamp())
633-
client_id = os.environ.get('OPENAI_CODEX_CLIENT_ID', '')
634-
635-
# All three accounts in preferred order. Account-1 is primary; 3 and 2 are
636-
# fallbacks only used when account-1's tokens are expired AND unrefreshable.
637-
candidates = [
638-
('', os.environ.get('OPENAI_CODEX_ACCESS_TOKEN', ''), os.environ.get('OPENAI_CODEX_REFRESH_TOKEN', ''), os.environ.get('OPENAI_CODEX_ID_TOKEN', '')),
639-
('3', os.environ.get('OPENAI_CODEX_ACCESS_TOKEN_3', ''), os.environ.get('OPENAI_CODEX_REFRESH_TOKEN_3', ''), os.environ.get('OPENAI_CODEX_ID_TOKEN_3', '')),
640-
('2', os.environ.get('OPENAI_CODEX_ACCESS_TOKEN_2', ''), os.environ.get('OPENAI_CODEX_REFRESH_TOKEN_2', ''), ''),
641-
]
550+
return access, refresh, id_tok
642551
643552
chosen = None
644-
for (suffix, access, refresh, id_tok) in candidates:
645-
label = f'account-{"1" if suffix == "" else suffix}'
646-
if not access and not refresh:
647-
print(f'{label}: no tokens configured, skipping')
553+
for label in ('1', '2', '3'):
554+
rec = read_pod_auth(f'/chatgpt-auth/auth-{label}.json')
555+
if not rec:
648556
continue
649-
exp = token_exp(access) if access else 0
650-
# Prefer valid access token (with 60s buffer to avoid edge-race expiry)
651-
if access and exp > now_ts + 60:
652-
chosen = (suffix, access, refresh, id_tok, exp)
653-
print(f'{label}: access_token valid (expires {exp}, +{exp - now_ts}s)')
654-
break
655-
# Otherwise try refresh
656-
if refresh and client_id:
657-
print(f'{label}: access_token expired/missing, refreshing…')
658-
refreshed = refresh_token(refresh, client_id, label)
659-
if refreshed:
660-
new_access, new_refresh, new_id, new_exp = refreshed
661-
print(f'{label}: refreshed successfully (expires {new_exp}, +{new_exp - now_ts}s)')
662-
chosen = (suffix, new_access, new_refresh, new_id or id_tok, new_exp)
663-
break
664-
else:
665-
print(f'{label}: no refresh_token or client_id, skipping')
666-
667-
if not chosen:
668-
with open('/chatgpt-auth/auth.json', 'w') as f:
669-
json.dump({}, f)
670-
print('No valid Codex token found after checking all accounts — wrote empty auth.json')
671-
sys.exit(0)
672-
673-
suffix, access, refresh, id_tok, exp = chosen
674-
out = {'access_token': access, 'expires_at': exp}
675-
if refresh:
676-
out['refresh_token'] = refresh
677-
if id_tok:
678-
out['id_token'] = id_tok
557+
access, refresh, id_tok = rec
558+
chosen = (label, access, refresh, id_tok, token_exp(access))
559+
print(f'seed: using pod-side auth-{label}.json (exp={chosen[4]})')
560+
break
561+
562+
out = {}
563+
if chosen:
564+
label, access, refresh, id_tok, exp = chosen
565+
out = {'access_token': access, 'expires_at': exp}
566+
if refresh:
567+
out['refresh_token'] = refresh
568+
if id_tok:
569+
out['id_token'] = id_tok
570+
else:
571+
print('seed: no pod-side auth-N.json found — writing empty auth.json. '
572+
'Run `/scripts/auth-login.sh <N>` from the codex-cli sidecar to seed.')
679573
680574
with open('/chatgpt-auth/auth.json', 'w') as f:
681575
json.dump(out, f)
682-
683-
exp_str = datetime.datetime.fromtimestamp(exp, datetime.timezone.utc).isoformat()
684-
print(f'auth.json written — account-{"1" if suffix == "" else suffix} expires_at={exp} ({exp_str})')
685-
env:
686-
- name: OPENAI_CODEX_ACCESS_TOKEN
687-
valueFrom:
688-
secretKeyRef:
689-
name: api-keys
690-
key: openai-codex-access-token
691-
optional: true
692-
- name: OPENAI_CODEX_REFRESH_TOKEN
693-
valueFrom:
694-
secretKeyRef:
695-
name: api-keys
696-
key: openai-codex-refresh-token
697-
optional: true
698-
- name: OPENAI_CODEX_ID_TOKEN
699-
valueFrom:
700-
secretKeyRef:
701-
name: api-keys
702-
key: openai-codex-id-token
703-
optional: true
704-
- name: OPENAI_CODEX_ACCESS_TOKEN_3
705-
valueFrom:
706-
secretKeyRef:
707-
name: api-keys
708-
key: openai-codex-access-token-3
709-
optional: true
710-
- name: OPENAI_CODEX_REFRESH_TOKEN_3
711-
valueFrom:
712-
secretKeyRef:
713-
name: api-keys
714-
key: openai-codex-refresh-token-3
715-
optional: true
716-
- name: OPENAI_CODEX_ID_TOKEN_3
717-
valueFrom:
718-
secretKeyRef:
719-
name: api-keys
720-
key: openai-codex-id-token-3
721-
optional: true
722-
# Account-2: used when accounts 1 and 3 are exhausted/expired
723-
- name: OPENAI_CODEX_ACCESS_TOKEN_2
724-
valueFrom:
725-
secretKeyRef:
726-
name: api-keys
727-
key: openai-codex-access-token-2
728-
optional: true
729-
- name: OPENAI_CODEX_REFRESH_TOKEN_2
730-
valueFrom:
731-
secretKeyRef:
732-
name: api-keys
733-
key: openai-codex-refresh-token-2
734-
optional: true
735-
# Client ID for OAuth refresh flow
736-
- name: OPENAI_CODEX_CLIENT_ID
737-
valueFrom:
738-
secretKeyRef:
739-
name: api-keys
740-
key: openai-codex-client-id
741-
optional: true
742576
volumeMounts:
743577
- name: chatgpt-auth
744578
mountPath: /chatgpt-auth

0 commit comments

Comments
 (0)