Skip to content

feat: cut standalone /plan-eng-review; /autoplan keeps its eng phase (GST-1698) - #4108

Merged
capy-ai[bot] merged 5 commits into
mainfrom
capy/gst-1698-cut-plan-eng-review
Nov 7, 2026
Merged

capy-ai[bot] merged 5 commits into
mainfrom
capy/gst-1698-cut-plan-eng-review

Conversation

@garrytan

@garrytan garrytan commented Nov 7, 2026 •

Copy link
Copy Markdown
Owner
{
  "schema_version": 1,
  "pr": 4108,
  "head": "6cfe8b6d16e6795ca3f1ad52d174e12c2f8e2f32",
  "version": "1.97.0.0",
  "base": "main",
  "gate": {
    "test": {
      "pass": 1621,
      "fail": 0,
      "skip": 0
    }
  },
  "ci_carried": [
    {
      "lane": "test",
      "run": 42334363210,
      "attempt": 1,
      "tree": "3a0838b13134d556ecb902a6c1b1342e4d4e6dd7"
    }
  ],
  "session_kind": "unattended",
  "head_tree": "b442e570d48af1437888a70485a6664e75cdd318",
  "product_patch_id": "f12cd3fe820c2b8d90cfa2b13230b4dd941d3cc0",
  "gstack_version": "1.97.0.0",
  "review_depth": {
    "depth": "light",
    "class": "code",
    "runtime_lines": 163,
    "rules": "70895942bb8a8ee2",
    "base": "1970da59c4dc5a56109acb0368ce3113a6e7dd09",
    "registry": "1970da59c4dc5a56109acb0368ce3113a6e7dd09",
    "gate_scan": {
      "tagged": 0,
      "head": "6cfe8b6d16e6795ca3f1ad52d174e12c2f8e2f32"
    },
    "reasons": [
      "runtime=163"
    ],
    "risk": [],
    "mode": "panel"
  },
  "pregate": "requires-remote cleared @b442e570d48a (8 checks, current for 6cfe8b6d16e6)",
  "requires_remote": [
    "free-tests.yml/cso-macos-launcher: https://github.com/garrytan/gstack/actions/runs/42336829460/attempts/1",
    "free-tests.yml/macos-named-regressions: https://github.com/garrytan/gstack/actions/runs/42336829460/attempts/1",
    "windows-setup-e2e.yml/windows-setup: https://github.com/garrytan/gstack/actions/runs/42336275383/attempts/1",
    "windows-smoke.yml/windows-smoke: https://github.com/garrytan/gstack/actions/runs/42336275464/attempts/1",
    "pregate.json/remote_gate: remote-gate tree=b442e570d48a tests=36086 skipped=111 runs=42336829460/1 profiles=ci-reuse/free-tests heads=6cfe8b6d16e6 verified=2026-11-07T04:11:21.247Z"
  ],
  "issues": {
    "fixes": [],
    "refs": [],
    "flakes": []
  },
  "docs": {
    "status": "current",
    "scope": "README.md, AGENTS.md, CLAUDE.md, ARCHITECTURE.md and docs/ (tracked and new .md, .tmpl and .txt), base e325b6d to head 692c8e6 plus unstaged TODOS.md and the changelog fragment"
  },
  "review_loop": {
    "passes": 2,
    "blocking_found": 2,
    "deferred": 1,
    "noted": 0,
    "fixed": 1,
    "scope": "frozen",
    "fix_checks": 1,
    "rounds": 3,
    "ledgers_read": 3,
    "dispatches": {
      "specialist": 1,
      "red-team": 1
    }
  },
  "carry": {
    "from": "8cadbdc985a6259842722f58c82f370f985fd39b",
    "reason": "scoped re-check by anthropic/claude-opus-5-5 (range-diff 28 lines) + test-only change + release metadata + regenerated outputs",
    "product_patch_id": "f12cd3fe820c2b8d90cfa2b13230b4dd941d3cc0",
    "changed_paths": [
      "generated:autoplan/SKILL.md",
      "product:autoplan/SKILL.md.tmpl",
      "metadata:changelog.d/capy-gst-1698-cut-plan-eng-review.md",
      "generated:gstack/llms.txt",
      "test:test/fixtures/skill-md-size-allowlist.json"
    ],
    "merge_commits": 0,
    "range_diff": "1:  384fa5b0 = 1:  384fa5b0 feat: cut standalone /plan-eng-review; /autoplan keeps its eng phase (GST-1698)\n2:  0124a2c4 = 2:  0124a2c4 test: lower shrunk SKILL.md size pins; keep the retired methodology's pins measured beside /autoplan\n3:  692c8e6b = 3:  692c8e6b fix: carry /autoplan's plan-eng-review.md into Kiro installs\n4:  8cadbdc9 = 4:  8cadbdc9 docs: file /plan-eng-review retirement follow-ups; scope the changelog claim to ./setup\n-:  -------- > 5:  6cfe8b6d fix: route architecture-review requests to /autoplan by its description (journey-plan-eng)",
    "covers": [
      "pre-landing review",
      "adversarial review",
      "gate integrity",
      "documentation audit"
    ],
    "reruns": [
      "CI at head (gstack-ship-queue ready requires it); test and helper code changed since the reviewed head runs there unreviewed"
    ],
    "ancestry": "linear",
    "trees": [
      "3a0838b13134d556ecb902a6c1b1342e4d4e6dd7"
    ]
  }
}
{"head":"8cadbdc985a6259842722f58c82f370f985fd39b","version":"1.97.0.0","gstack_version":"1.97.0.0","review_loop":{"passes":2,"blocking_found":2,"deferred":1,"noted":0,"fixed":1,"scope":"frozen","fix_checks":1,"rounds":3,"ledgers_read":3,"dispatches":{"specialist":1,"red-team":1}},"depth":"light"}

What changed

/plan-eng-review no longer installs as a standalone skill on any host. Its methodology stays in plan-eng-review/ and still runs as /autoplan's Phase 3, byte for byte.

  • Retired list. scripts/discover-skills.ts exports RETIRED_SKILLS = ['plan-eng-review']. setup, bin/gstack-relink and bin/gstack-install-registry.sh treat it as disabled and can't re-enable it. The next ./setup or upgrade removes a gstack-owned plan-eng-review / gstack-plan-eng-review link, and leaves a user's own same-name skill alone. disabled_skills output and counts show only what the user disabled.
  • External hosts. The Codex, Factory, OpenCode and other external renders now write the methodology to gstack-autoplan/plan-eng-review.md instead of a nested SKILL.md, because those hosts register every nested SKILL.md. It has no agents/openai.yaml and no hoisted preamble. On all 11 external hosts the file is byte-identical to main's gstack-plan-eng-review/SKILL.md. composition.ts and gstack-autoplan-snapshot resolve that path. Kiro installs copy files one by one, so setup's Kiro loop now also carries generated top-level siblings of SKILL.md, which puts plan-eng-review.md beside gstack-autoplan/SKILL.md there too (a /ship review fix, 692c8e6).
  • Redirects to /autoplan. These now point to /autoplan: the router, CLAUDE.md, gstack-skill-start routing, the office-hours handoff, and the CEO, design and devex chaining (which no longer says "required gate"). So do the qa, retro, spec and test-audit mentions, the review dashboard's Eng Review row, and /ship's over-200-lines hint. /autoplan adds "Next Steps — Review Chaining" to its skip list so it doesn't recurse. Its guard-failure fallback now names /plan-ceo-review then /plan-devex-review.
  • Routing fix (6cfe8b6). After the cut, no installed skill description matched an architecture review request, so the paid routing case journey-plan-eng (periodic, kind rule) failed twice at 8cadbdc with 0 Skill calls. /autoplan's description now names the eng phase's architecture, edge cases, failure modes and tests, and its template adds the triggers "review the architecture" and "engineering review". The wording is shorter, so autoplan/SKILL.md doesn't grow: the 12 gstack-autoplan pins in test/fixtures/skill-md-size-allowlist.json went down (11 hosts by 2 chars, hermes by 51). The renders and gstack/llms.txt are regenerated, and the changelog fragment gained one clause.
  • Docs. docs/skills.md replaces the section with a "cut" note that carries the eval numbers. README, AGENTS.md, hosts.md, PROJECT_STRUCTURE, OPENCLAW, voice-input, troubleshooting and ARCHITECTURE are updated, and there's a CHANGELOG fragment, which now says ./setup no longer installs the skill on any host. TODOS.md files the review's follow-ups.

Why

Two evals found no gain from the standalone skill. In the seeded-flaw panel, plain Opus 5.5 found the planted flaws in 15 of 16 reviews. In the wave 1 outcome test, it built 22 of 22 features, and a session told to use the skill cost 7.1x a plain one (GST-1218, gstack-evals#253). Garry approved the cut on GEVA-2's R41 list ("A1 cut /plan-eng-review"). On GST-1698 he chose options A plus B, which mean an unregistered methodology source, retirement through disabled_skills, and an external copy beside gstack-autoplan.

Blast radius

  • /autoplan Phase 3 is unchanged. In a fixed-plan fixture (gstack-autoplan next --full --ui --developer-tool on the simulated Capy host), the eng methodology.md sha 539f595b… is identical on main and this branch. The only difference in methodology.json is the envelope byte offsets, because the checkout root paths differ by 17 bytes. The per-source sha values are equal. The CEO, design and devex methodologies differ only in the redirected chaining and dashboard lines.
  • /ship gating is unchanged. Eng Review still accepts review or autoplan's plan-eng-review log row, and a missing standalone skill never blocks.
  • The PTY/e2e register action still copies plan-eng-review/ from the source tree, so the methodology cases keep testing what Phase 3 loads.
  • The size ledger records ship at −21 bytes. No ratchet, budget or threshold was loosened.
  • Kiro: before the fix, ./setup --host kiro in a fixture HOME left ~/.kiro/skills/gstack-autoplan/ with only SKILL.md and sections/, so /autoplan Phase 3 pointed at a missing file. After it, the same install has plan-eng-review.md linked to the render (exploratory QA, captures below). The new loop uses the same ownership guards as the existing sections/ loop, so a user's own file or a foreign link next to SKILL.md is kept.

Tests

  • New test/retired-skills.test.ts covers stale-link removal by relink (plain and prefixed) and by setup on external hosts, a user-owned same-name skill left alone, the shared retired list in setup/relink and the generator, and the external render path. It fails first on main and passes on this branch.
  • Updated these existing tests to the new paths and labels, with methodology-content assertions kept: gen-skill-docs, gen-skill-docs-checks, skill-validation, doctor-components, eng-review-routing, disabled-plan-review-evidence, plan-review-cases, autoplan-methodology-names, autoplan-review-discovery, shared-libs-rendering, outside-voice-routing, external-host-carve, the autoplan guard/journal/publication tests, and skill-routing-e2e.

Approvals

Approved: yes (record r-9500a191f09b). The full approval record stays private: this repository is public or its visibility is unknown.

  • Garry, 2026-11-06: "do it" on R41 A1 (cut /plan-eng-review).
  • Garry on GST-1698: "A plus B … Build."

Summary

Two evals showed the standalone /plan-eng-review added nothing, so it stops installing as a skill and /autoplan keeps running its methodology as Phase 3, byte for byte. Everything that used to send people to it (the router, the office-hours handoff, the CEO, design and devex chaining, the dashboard and /ship's big-diff hint) now points at /autoplan, and existing installs lose the old entry on the next setup. The /ship review caught that Kiro copies skills file by file and would have lost the Phase 3 file, so that is fixed here with a failing-first test.

Test Coverage

  • New test/retired-skills.test.ts covers stale-link removal by relink (plain and prefixed) and by setup on external hosts, a user-owned same-name skill left alone, the shared retired list in setup/relink and the generator, and the external render path. It fails first on main and passes on this branch.
  • Updated these existing tests to the new paths and labels, with methodology-content assertions kept: gen-skill-docs, gen-skill-docs-checks, skill-validation, doctor-components, eng-review-routing, disabled-plan-review-evidence, plan-review-cases, autoplan-methodology-names, autoplan-review-discovery, shared-libs-rendering, outside-voice-routing, external-host-carve, the autoplan guard/journal/publication tests, and skill-routing-e2e.
  • test/setup-kiro-native.test.ts now seeds a generated gstack-autoplan/plan-eng-review.md and asserts the Kiro install carries it. Regression proof: fails at 0124a2c (0 pass, 2 fail) · passes after fix (with retired-skills and setup-prune-stale-generated: 12 pass, 0 fail).

Tests: 1746 → 1746 (+0 new files; 1 existing test extended)
Coverage: 33% value-weighted (67% including 1 weakly covered path)
Test value: 0 tests written, 1 rejected by the authoring gate, 1 existing test extended, 1 path weakly covered (weak = ★, gate-failing or unrated).

Proposed tests (filed in TODOS.md): generateLlmsTxt lists no plan-eng-review entry (protects: a retired skill never appears in llms.txt; fails when the RETIRED_SKILLS filter is removed); HYBRID_SKILLS holds none of RETIRED_SKILLS (protects: OpenClaw hybrid mode never registers a retired skill). Rejected: lib/ship-preflight.ts DIFF hint (prose-only string change).

Tests ran: test carried from CI (free-tests.yml at the final head; see the receipt)

Pre-Landing Review

Pre-Landing Review: 8 issues — 1 fixed (CRITICAL setup:4018 Kiro installs lacked /autoplan's Phase 3 file; fixed in 692c8e6, fix-check clean), 2 skipped, 1 deferred, 4 noted.

  • Skipped (pass 1, below the floor): lib/harness-register.ts:236 dots/generic harness still register the retired skill (re-raised in pass 2 and deferred below); scripts/gen-skill-docs.ts:888 external-host preamble pointer names the pruned directory (fixing it changes Phase 3 bytes, which this PR pins byte-identical to main).

REVIEW_LOOP: passes=2 blocking_found=2 deferred=1 noted=0 fixed=1 scope=frozen ledgers_read=3 fix_checks=1 dispatches=2

Gate edits: none detected (52 listed, 35 read; patterns: RH-1,RH-3,RH-4,RH-12,RH-13,RH-14,RH-15,RH-16; unlisted idioms: none)

Outside review: openai/gpt-6.1-sol (host subagent), completed: request changes for the dots/generic harness registration (labelled blocking). The lead kept its CRITICAL label; review scope classified it outside the diff and deferred it, and by the owner's blocking floor it fails no test, breaks no install and loses no data, because the retired skill's files still exist and still work. Native adversarial: ship as-is, four non-blocking notes.

Deferred findings

Blocking findings in code this PR did not change (review scope: frozen); filed as issues and not closed by this PR.

  • Deferred: unfiled (dry run) — CRITICAL lib/harness-register.ts:236 dots and generic (claude-render) harness targets still register and, in all mode, install the retired plan-eng-review skill. Not filed as a GitHub issue (unattended run; crew rules keep gh writes off garrytan's identity); tracked in TODOS.md "P2: /plan-eng-review retirement follow-ups".

Review notes

Below the blocking floor (advisory, informational, test-gap or wording items); recorded here and not fixed in this PR.

  • INFORMATIONAL lib/harness-register.ts:236 dots and generic harness targets still register plan-eng-review (same as outside-voice#1); Blocks: no
  • INFORMATIONAL scripts/gen-skill-docs.ts:888 external-host plan-eng-review.md shared-preamble pointer names the pruned gstack-plan-eng-review directory
  • INFORMATIONAL bin/gstack-doctor-components.sh:1 gstack-doctor --for plan-eng-review now exits 2 unknown skill with no redirect to /autoplan
  • INFORMATIONAL hosts/qm.ts:1 QM edition still builds gstack-plan from plan-eng-review sources (product call)
  • INFORMATIONAL autoplan/SKILL.md.tmpl:10 (scoped re-check of 6cfe8b6) the new "review the architecture" and "engineering review" trigger phrases sit only in the template's long description; the rendered frontmatter keeps the first sentence, so routing rests on its "eng (architecture, edge cases, failure modes, tests)" wording, which journey-plan-eng passed at 6cfe8b6; Blocks: no

Exploratory QA

Surface: functional (./setup --host kiro in a fixture HOME). Pass 1 at 0124a2c: the install held only SKILL.md and sections/ beside gstack-autoplan (PHASE3_MISSING), confirming the finding; after the fix the replay linked plan-eng-review.md to the render, and a second run kept it. That pass ended inconclusive because its pre-fix captures were superseded. Pass 2 at 692c8e6 on a fresh fixture: pass (no gstack-plan-eng-review entry, plan-eng-review.md present). Not probed: other external hosts, which link whole skill directories and are covered by test/retired-skills.test.ts.

Design Review

No frontend files changed — design review skipped.

Eval Results

Evals: none run here. The change moves skill prose and generated renders; /autoplan Phase 3 bytes are unchanged (see Blast radius), and paid evals are the Captain's gate.

  • journey-plan-eng (skill-routing-e2e, periodic, kind rule): FAIL 0/1 twice at 8cadbdc (session completed with 0 Skill calls, so the routing check failed), then PASS 1/1 at 6cfe8b6 after the routing fix ($0.06, reported by the lane). Command: bun run scripts/test-paid-shards.ts --tier periodic --case journey-plan-eng.

Review bot comments

No review bot comments.

Scope Drift

Scope Check: CLEAN

Plan Completion

Plan completion audit: not run (no plan is bound to this branch and no docs/designs/ file matches). Fix: add "Plan: " to the PR body, or run /autoplan.

TODOS

No TODO items completed in this PR. Added "P2: /plan-eng-review retirement follow-ups" (five items from this review).

Documentation

Status: current. No authored-doc edits were needed.

Scope: base e325b6d through head 692c8e6 (3 commits, GST-1698, which cuts standalone /plan-eng-review), plus the unstaged changes to TODOS.md and the changelog fragment. Discovery covered tracked and new .md, .tmpl and .txt docs, searched for every remaining /plan-eng-review reference.

Documentation health:

  • README.md: current (the /autoplan row still describes the eng phase correctly)
  • docs/skills.md: updated on this branch (a cut notice that points to /autoplan)
  • docs/PROJECT_STRUCTURE.md, docs/hosts.md, docs/OPENCLAW.md, docs/troubleshooting.md, docs/autoplan-guard-troubleshooting.md, docs/voice-input.md, docs/test-value-bar.md: updated on this branch
  • docs/inventory.md: stale but generated. It still lists plan-eng-review, so regenerate it with bin/gstack-inventory (parent-owned).
  • Historical docs (ON_THE_LOC_CONTROVERSY, test-audit-2026-09/10, askuserquestion-split): current as records

Debt: no coverage gaps or diagram drift found. The open retirement follow-ups are already filed in TODOS.md (P2/P3).

Test plan

  • bun test --timeout=30000 test/setup-kiro-native.test.ts test/retired-skills.test.ts test/setup-prune-stale-generated.test.ts: 12 pass, 0 fail
  • Scoped re-check of 8cadbdc..6cfe8b6 (one reader, anthropic/claude-opus-5-5): NO FINDINGS. The reader read the 54-line packet (range_diff_sha256 26513c03); once the pregate regen check proved autoplan/SKILL.md and gstack/llms.txt as regenerated outputs, the carry packet dropped those two hunks to 28 lines (8ca04233), every line of which was in the packet the reader cleared. Its focused checks: skill-md-size-guard, catalog-budget, context-budget-ratchet, eng-review-routing, catalog-trim, instruction-facts and retired-skills all green (65 tests, no failures), and parity-suite green (2 tests); gen:skill-docs --dry-run fresh.
  • bun run test: carried from CI at the final head (free-tests.yml; run id in the receipt)
  • pregate guards at 8cadbdc's tree: typecheck, skill-size, parity-suite, context-budget, module-size, goldens, doc-caps, conflict-markers all pass

🤖 Generated with Claude Code

Open in Capy

…(GST-1698)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
@trunk-io

trunk-io Bot commented Nov 7, 2026 •

Copy link
Copy Markdown

😎 This pull request was merged.

garrytan and others added 4 commits November 7, 2026 02:55
… pins measured beside /autoplan

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Kiro's per-skill loop copied only SKILL.md and sections, so ~/.kiro/skills/gstack-autoplan lacked the Phase 3 methodology file its SKILL.md reads. Generated sibling *.md files now travel with SKILL.md under the same ownership guards.

Review-Finding: p1/specialist-batched#1 reader=red-team
Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…g claim to ./setup

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
…on (journey-plan-eng)

Co-authored-by: capy-ai[bot] <230910855+capy-ai[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant