Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ All notable changes to this project will be documented in this file.

## [Unreleased]

### Added

- New `codex/gpt-5.3-codex` model support for Codex CLI (ChatGPT subscription)
- Updated default Codex CLI model from `gpt-5.2-codex` to `gpt-5.3-codex`

### Fixed

- Added `--skip-git-repo-check` flag to Codex CLI calls for non-git directory support
Expand Down
17 changes: 12 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,7 @@ You describe product --> Claude drafts spec --> Multiple LLMs critique in parall
| Mistral | `MISTRAL_API_KEY` | `mistral/mistral-large`, `mistral/codestral` |
| Groq | `GROQ_API_KEY` | `groq/llama-3.3-70b-versatile` |
| OpenRouter | `OPENROUTER_API_KEY` | `openrouter/openai/gpt-4o`, `openrouter/anthropic/claude-3.5-sonnet` |
| Codex CLI | ChatGPT subscription | `codex/gpt-5.2-codex`, `codex/gpt-5.1-codex-max` |
| Codex CLI | ChatGPT subscription | `codex/gpt-5.3-codex`, `codex/gpt-5.2-codex` |
| Gemini CLI | Google account | `gemini-cli/gemini-3-pro-preview`, `gemini-cli/gemini-3-flash-preview` |
| Deepseek | `DEEPSEEK_API_KEY` | `deepseek/deepseek-chat` |
| Zhipu | `ZHIPUAI_API_KEY` | `zhipu/glm-4`, `zhipu/glm-4-plus` |
Expand Down Expand Up @@ -140,7 +140,7 @@ See the full model list at [openrouter.ai/models](https://openrouter.ai/models).
npm install -g @openai/codex

# Use Codex models (prefix with codex/)
python3 debate.py critique --models codex/gpt-5.2-codex,gemini/gemini-2.0-flash < spec.md
python3 debate.py critique --models codex/gpt-5.3-codex,gemini/gemini-2.0-flash < spec.md
```

**Reasoning effort:**
Expand All @@ -149,14 +149,21 @@ Control how much thinking time the model uses with `--codex-reasoning`:

```bash
# Available levels: low, medium, high, xhigh (default: xhigh)
python3 debate.py critique --models codex/gpt-5.2-codex --codex-reasoning high < spec.md
python3 debate.py critique --models codex/gpt-5.3-codex --codex-reasoning high < spec.md
```

Higher reasoning effort produces more thorough analysis but uses more tokens.
| Level | Description |
|----------|---------------------------------------------------|
| `low` | Fast responses with lighter reasoning |
| `medium` | Balances speed and reasoning depth (Codex default) |
| `high` | Greater reasoning depth for complex problems |
| `xhigh` | Extra high reasoning depth (adversarial-spec default) |

The plugin defaults to `xhigh` because adversarial spec review benefits from thorough, deep analysis. Use `medium` or `high` for faster iterations.

**Available Codex models:**
- `codex/gpt-5.3-codex` - GPT-5.3 via Codex CLI
- `codex/gpt-5.2-codex` - GPT-5.2 via Codex CLI
- `codex/gpt-5.1-codex-max` - GPT-5.1 Max via Codex CLI

Check Codex CLI installation status:

Expand Down
6 changes: 3 additions & 3 deletions skills/adversarial-spec/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,12 +30,12 @@ Generate and refine specifications through iterative debate with multiple LLMs u
| OpenRouter | `OPENROUTER_API_KEY` | `openrouter/openai/gpt-4o`, `openrouter/anthropic/claude-3.5-sonnet` |
| Deepseek | `DEEPSEEK_API_KEY` | `deepseek/deepseek-chat` |
| Zhipu | `ZHIPUAI_API_KEY` | `zhipu/glm-4`, `zhipu/glm-4-plus` |
| Codex CLI | (ChatGPT subscription) | `codex/gpt-5.2-codex`, `codex/gpt-5.1-codex-max` |
| Codex CLI | (ChatGPT subscription) | `codex/gpt-5.3-codex`, `codex/gpt-5.2-codex` |
| Gemini CLI | (Google account) | `gemini-cli/gemini-3-pro-preview`, `gemini-cli/gemini-3-flash-preview` |

**Codex CLI Setup:**
- Install: `npm install -g @openai/codex && codex login`
- Reasoning effort: `--codex-reasoning` (minimal, low, medium, high, xhigh)
- Reasoning effort: `--codex-reasoning` (low, medium, high, xhigh). Default: `xhigh` for thorough spec analysis. Codex model default is `medium`.
- Web search: `--codex-search` (enables web search for current information)

**Gemini CLI Setup:**
Expand Down Expand Up @@ -340,7 +340,7 @@ Then present available models to the user using AskUserQuestion with multiSelect
- `zhipu/glm-4-plus` - Enhanced GLM model

**If Codex CLI is installed, include:**
- `codex/gpt-5.2-codex` - OpenAI Codex with extended reasoning
- `codex/gpt-5.3-codex` - OpenAI Codex with extended reasoning

**If Gemini CLI is installed, include:**
- `gemini-cli/gemini-3-pro-preview` - Google Gemini 3 Pro
Expand Down
4 changes: 2 additions & 2 deletions skills/adversarial-spec/scripts/debate.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
Usage:
echo "spec" | python3 debate.py critique --models gpt-4o
echo "spec" | python3 debate.py critique --models gpt-4o,gemini/gemini-2.0-flash,xai/grok-3 --doc-type prd
echo "spec" | python3 debate.py critique --models codex/gpt-5.2-codex,gemini/gemini-2.0-flash --doc-type tech
echo "spec" | python3 debate.py critique --models codex/gpt-5.3-codex,gemini/gemini-2.0-flash --doc-type tech
echo "spec" | python3 debate.py critique --models gpt-4o --focus security
echo "spec" | python3 debate.py critique --models gpt-4o --persona "security engineer"
echo "spec" | python3 debate.py critique --models gpt-4o --context ./api.md --context ./schema.sql
Expand All @@ -28,7 +28,7 @@
Mistral: MISTRAL_API_KEY models: mistral/mistral-large, etc.
Groq: GROQ_API_KEY models: groq/llama-3.3-70b, etc.
OpenRouter: OPENROUTER_API_KEY models: openrouter/openai/gpt-4o, openrouter/anthropic/claude-3.5-sonnet, etc.
Codex CLI: (ChatGPT subscription) models: codex/gpt-5.2-codex, codex/gpt-5.1-codex-max
Codex CLI: (ChatGPT subscription) models: codex/gpt-5.3-codex, codex/gpt-5.2-codex
Install: npm install -g @openai/codex && codex login
Reasoning: --codex-reasoning xhigh (minimal, low, medium, high, xhigh)

Expand Down
5 changes: 3 additions & 2 deletions skills/adversarial-spec/scripts/providers.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
"zhipu/glm-4": {"input": 1.40, "output": 1.40},
"zhipu/glm-4-plus": {"input": 7.00, "output": 7.00},
# Codex CLI models (uses ChatGPT subscription, no per-token cost)
"codex/gpt-5.3-codex": {"input": 0.0, "output": 0.0},
"codex/gpt-5.2-codex": {"input": 0.0, "output": 0.0},
"codex/gpt-5.1-codex-max": {"input": 0.0, "output": 0.0},
"codex/gpt-5.1-codex-mini": {"input": 0.0, "output": 0.0},
Expand Down Expand Up @@ -308,7 +309,7 @@ def list_providers():
# Codex CLI (uses ChatGPT subscription, not API key)
codex_status = "[installed]" if CODEX_AVAILABLE else "[not installed]"
print(f" {'Codex CLI':12} {'(ChatGPT subscription)':24} {codex_status}")
print(" Example models: codex/gpt-5.2-codex, codex/gpt-5.1-codex-max")
print(" Example models: codex/gpt-5.3-codex, codex/gpt-5.2-codex")
print(
" Reasoning: --codex-reasoning (minimal, low, medium, high, xhigh)"
)
Expand Down Expand Up @@ -382,7 +383,7 @@ def get_available_providers() -> list[tuple[str, Optional[str], str]]:

# Add Codex CLI if available
if CODEX_AVAILABLE:
available.append(("Codex CLI", None, "codex/gpt-5.2-codex"))
available.append(("Codex CLI", None, "codex/gpt-5.3-codex"))

# Add Gemini CLI if available
if GEMINI_CLI_AVAILABLE:
Expand Down
34 changes: 34 additions & 0 deletions skills/adversarial-spec/scripts/tests/test_models.py
Original file line number Diff line number Diff line change
Expand Up @@ -102,6 +102,13 @@ def test_default_values(self):
assert tracker.total_cost == 0.0 # Must be exactly 0.0
assert tracker.by_model == {}

def test_codex_gpt53_zero_cost(self):
tracker = CostTracker()
cost = tracker.add("codex/gpt-5.3-codex", 10000, 5000)
# Codex CLI models have $0 per-token cost
assert cost == 0.0
assert tracker.total_cost == 0.0

def test_tracks_by_model(self):
tracker = CostTracker()
tracker.add("gpt-4o", 1000, 500)
Expand Down Expand Up @@ -540,6 +547,22 @@ def test_extracts_model_name_from_codex_prefix(self, mock_run):
assert "gpt-5.2-codex" in cmd
assert "codex/gpt-5.2-codex" not in cmd

@patch("models.CODEX_AVAILABLE", True)
@patch("models.subprocess.run")
def test_extracts_gpt53_codex_model_name(self, mock_run):
mock_run.return_value = Mock(
returncode=0,
stdout='{"type":"item.completed","item":{"type":"agent_message","text":"Response"}}\n{"type":"turn.completed","usage":{"input_tokens":200,"output_tokens":100}}',
stderr="",
)
response, inp, out = call_codex_model("sys", "user", "codex/gpt-5.3-codex")
cmd = mock_run.call_args[0][0]
assert "gpt-5.3-codex" in cmd
assert "codex/gpt-5.3-codex" not in cmd
assert response == "Response"
assert inp == 200
assert out == 100

@patch("models.CODEX_AVAILABLE", True)
@patch("models.subprocess.run")
def test_parses_jsonl_response(self, mock_run):
Expand Down Expand Up @@ -980,6 +1003,17 @@ def test_routes_codex_model_to_handler(self, mock_codex):
mock_codex.assert_called_once()
assert result.model == "codex/gpt-5"

@patch("models.call_codex_model")
@patch("models.CODEX_AVAILABLE", True)
def test_routes_gpt53_codex_to_handler(self, mock_codex):
mock_codex.return_value = ("[AGREE]\n[SPEC]spec[/SPEC]", 200, 100)

result = call_single_model("codex/gpt-5.3-codex", "spec", 1, "prd")
mock_codex.assert_called_once()
assert result.model == "codex/gpt-5.3-codex"
assert result.agreed is True
assert result.spec == "spec"

@patch("models.call_codex_model")
@patch("models.CODEX_AVAILABLE", True)
@patch("models.time.sleep")
Expand Down
19 changes: 19 additions & 0 deletions skills/adversarial-spec/scripts/tests/test_providers.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ def test_model_costs_has_expected_models(self):
"mistral/mistral-large",
"deepseek/deepseek-chat",
"zhipu/glm-4",
"codex/gpt-5.3-codex",
]
for model in expected:
assert model in MODEL_COSTS
Expand Down Expand Up @@ -742,6 +743,11 @@ def test_includes_codex_cli_when_available(self):
available = get_available_providers()
provider_names = [name for name, _, _ in available]
assert "Codex CLI" in provider_names
# Verify the default model is gpt-5.3-codex
for name, key, model in available:
if name == "Codex CLI":
assert model == "codex/gpt-5.3-codex"
assert key is None

def test_includes_gemini_cli_when_available(self):
from providers import get_available_providers
Expand Down Expand Up @@ -837,6 +843,19 @@ def test_validates_codex_availability(self):
assert valid == []
assert invalid == ["codex/gpt-5.2-codex"]

def test_validates_codex_gpt53_availability(self):
from providers import validate_model_credentials

with patch("providers.CODEX_AVAILABLE", True):
valid, invalid = validate_model_credentials(["codex/gpt-5.3-codex"])
assert valid == ["codex/gpt-5.3-codex"]
assert invalid == []

with patch("providers.CODEX_AVAILABLE", False):
valid, invalid = validate_model_credentials(["codex/gpt-5.3-codex"])
assert valid == []
assert invalid == ["codex/gpt-5.3-codex"]

def test_validates_gemini_cli_availability(self):
from providers import validate_model_credentials

Expand Down