Skip to content

feat: Codex-compatible skills distribution (spike)#88

Draft
adunn08 wants to merge 3 commits into
mainfrom
feat/port-codex
Draft

feat: Codex-compatible skills distribution (spike)#88
adunn08 wants to merge 3 commits into
mainfrom
feat/port-codex

Conversation

@adunn08

@adunn08 adunn08 commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Spike: a repeatable, dependency-free build step that produces a Codex-compatible distribution of the skills catalog under dist/codex/, plus a feasibility writeup. Lane was limited to scripts/build-codex.mjs and dist/codex/**; nothing else was touched.

Skill count

  • 102 skills in source (skills/<name>/SKILL.md); 102 emitted to dist/codex/.agents/skills/.
  • 488 reference files copied byte for byte (nested ref subtrees preserved).

Validation results

Run node scripts/build-codex.mjs -- all checks PASS:

  • [PASS] A. every emitted SKILL.md has non-empty name + description (102 ok)
  • [PASS] B. emitted skill count equals source (source=102 emitted=102)
  • [PASS] C. every source references file has an emitted counterpart
  • [PASS] D. no emitted frontmatter has keys beyond name/description

Frontmatter normalization

Every source skill carries 5 uniform keys. The build keeps the portable core (name, description) on each emitted SKILL.md and moves the rest -- category, catalog_summary, display_order -- into a per-skill sidecar references/_claude-frontmatter-extras.yaml. Fully reversible; SKILL.md bodies are left byte-identical.

MCP template note

dist/codex/agents/openai.yaml is a commented template listing detected MCP servers with placeholder url/name/transport fields for the operator to fill in. No real server config is fabricated. Detected: Ahrefs MCP (7 skills, the SEO audit suite), Similarweb, plus Chrome/Playwright/Windows/Linear/GitHub (integration-orchestrator examples), and 7 generic MCP mentions flagged for operator review.

Feasibility

Mechanically straightforward and low risk: a reversible, dependency-free transform yields a clean .agents/skills/ tree for all 102 skills, with the only real integration work being operator-supplied MCP server config.

See dist/codex/PORT_NOTES.md for the full writeup.

🤖 Generated with Claude Code

adunn08 added a commit that referenced this pull request Jun 1, 2026
Phase 2 of the Codex port spike. Extends scripts/build-codex.mjs and the
dist/codex distribution so PR #88 can come out of draft.

- Determinism: the build now builds twice and asserts byte-identical
  generated output (.agents/ + agents/openai.yaml).
- Drift guard: new --check flag rebuilds into a temp dir and diffs against
  the committed dist/codex, exiting nonzero on any drift.
- Codex 1024-char description cap: confirmed against codex-cli 0.118.0 that
  3 skills (creative-direction, integration-orchestrator, logo-design) failed
  to load because their descriptions exceeded 1024 chars. The build now emits
  a description truncated at a sentence boundary and preserves the full
  original in the sidecar (description_full). A gating check (E) prevents
  regression. Re-running the real CLI confirms all 102 skills now load with
  zero parse errors.
- Description-discrimination audit (heuristic): writes
  dist/codex/SKILL_DISCOVERY_AUDIT.md with a fixture table, low-confidence
  prompts, and flagged descriptions. Labeled heuristic throughout.
- openai.yaml sanity: asserts the file exists, every detected MCP server has a
  commented entry, and no uncommented urls/keys/tokens are present.
- PORT_NOTES.md updated with steps 1-4 results and an owner smoke-test recipe
  (notes Codex also reads ~/.codex/skills/).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@adunn08

adunn08 commented Jun 1, 2026

Copy link
Copy Markdown
Contributor Author

Phase 2: verification + hardening (automated results)

New commit e467201 extends the build and verifies discovery. Nothing outside the lane (scripts/build-codex.mjs, dist/codex/**) was touched.

Gating checks: all PASS

node scripts/build-codex.mjs:

  • Validation A-E PASS: name/description present, count 102 == 102, all references present, frontmatter clean, and (new) every emitted description within Codex's 1024-char cap.
  • Determinism PASS: building twice yields byte-identical generated output.
  • openai.yaml sanity PASS: file present; all 7 detected MCP servers (Ahrefs, Similarweb, Chrome, Playwright, Windows, Linear, GitHub) appear as commented entries; no uncommented urls/keys/tokens.

node scripts/build-codex.mjs --check PASS: committed dist/codex is in sync with a fresh rebuild (drift guard).

Real Codex CLI discovery (best effort, done)

Verified against installed codex-cli 0.118.0 (logged in). Loaded the emitted .agents/ tree via codex exec --cd <scratch> --skip-git-repo-check -s read-only with RUST_LOG=info:

  • Found a real blocker: 3 skills (creative-direction, integration-orchestrator, logo-design) originally failed to load with invalid description: exceeds maximum length of 1024 characters.
  • Fixed in-lane: the build now truncates over-cap descriptions at a sentence boundary and preserves the full original in the sidecar (description_full), fully reversible. After the fix, all 102 skills load with zero parse errors.
  • The subsequent model turn errored on an account/model mismatch (gpt-5.3-codex not allowed on a ChatGPT account) - unrelated to discovery, which completed first.

Discovery audit (heuristic)

dist/codex/SKILL_DISCOVERY_AUDIT.md: 6/8 fixture prompts place the expected skill in the top 3. 2 low-confidence (brand-archetype-system, experiment-design) from overlapping vocabulary; 3 skills lack an explicit "use when / triggers on" cue. Labeled heuristic throughout - it is a token-overlap proxy, not a runtime test.

One remaining gate before merge

The authoritative check is the owner's live smoke test (recipe in dist/codex/PORT_NOTES.md): drop .agents into a scratch Codex workspace, prompt "write a creative brief", and confirm creative-brief loads and shapes the output. Leaving this PR as a draft pending that.

adunn08 and others added 3 commits June 1, 2026 14:46
Add scripts/build-codex.mjs, a dependency-free Node ESM build step that
produces a Codex distribution of the 102-skill catalog under dist/codex/.

- Walks skills/<name>/ and copies each into dist/codex/.agents/skills/<name>/,
  preserving the references/ subtree byte for byte (488 files).
- Normalizes SKILL.md frontmatter to the portable core (name + description);
  moves category, catalog_summary, display_order into a per-skill sidecar
  (references/_claude-frontmatter-extras.yaml) so the port is reversible.
- Leaves SKILL.md bodies untouched.
- Detects MCP references and emits dist/codex/agents/openai.yaml as a commented
  wiring template (Ahrefs MCP powers the SEO audit suite; no real config is
  fabricated).
- Includes a validation pass (PASS/FAIL per check): non-empty name/description,
  skill-count parity, reference-file coverage, clean frontmatter.

Adds dist/codex/PORT_NOTES.md and dist/codex/README.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Phase 2 of the Codex port spike. Extends scripts/build-codex.mjs and the
dist/codex distribution so PR #88 can come out of draft.

- Determinism: the build now builds twice and asserts byte-identical
  generated output (.agents/ + agents/openai.yaml).
- Drift guard: new --check flag rebuilds into a temp dir and diffs against
  the committed dist/codex, exiting nonzero on any drift.
- Codex 1024-char description cap: confirmed against codex-cli 0.118.0 that
  3 skills (creative-direction, integration-orchestrator, logo-design) failed
  to load because their descriptions exceeded 1024 chars. The build now emits
  a description truncated at a sentence boundary and preserves the full
  original in the sidecar (description_full). A gating check (E) prevents
  regression. Re-running the real CLI confirms all 102 skills now load with
  zero parse errors.
- Description-discrimination audit (heuristic): writes
  dist/codex/SKILL_DISCOVERY_AUDIT.md with a fixture table, low-confidence
  prompts, and flagged descriptions. Labeled heuristic throughout.
- openai.yaml sanity: asserts the file exists, every detected MCP server has a
  commented entry, and no uncommented urls/keys/tokens are present.
- PORT_NOTES.md updated with steps 1-4 results and an owner smoke-test recipe
  (notes Codex also reads ~/.codex/skills/).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…s legal)

Rebased feat/port-codex onto main after PR #90 shortened the three over-cap
source descriptions. Rebuilt the distribution from the corrected catalog:

- creative-direction (900), integration-orchestrator (968), logo-design (976)
  are now under the 1024-char cap, so the truncation logic stays dormant: their
  emitted SKILL.md descriptions are the full source text and the description_full
  sidecar entries are gone.
- Skill count 102; --check PASS (committed dist matches a fresh rebuild);
  zero emitted descriptions exceed 1024 chars.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@adunn08 adunn08 force-pushed the feat/port-codex branch from e467201 to a1600f2 Compare June 1, 2026 20:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant