Skip to content

Commit 9b4279e

Browse files
feat(batch-bug-shepherd): operator visibility + fold-in invariant (#1451)
* feat(batch-bug-shepherd): operator visibility + fold-in invariant Refactors the batch-bug-shepherd skill to address two genesis-validated blockers and ship two missing capabilities discovered during a real sweep-all run over 14 microsoft/apm bugs: 1. OPERATOR VISIBILITY (was: silent 30-minute fan-outs) - New asset assets/progress-diagram.md: mermaid template + 5-state palette (pending/active/done/blocked/skipped) + per-phase render rules + dispatch-table contract. - SKILL.md adds 'Operator visibility is a contract' invariant; each phase boundary re-renders the diagram with current-phase coloring and prints a subagent_id -> target dispatch table BEFORE fan-out. - Operator can follow long sagas at a glance instead of waiting in the dark for the next checkpoint. 2. FOLD-IN INVARIANT (was: panel recommendations silently dropped) - assets/verdict-schema.json: shepherd_return gains required recommended_followups[] channel; completion_return gains folded_followups[] + deferred_followups[]; extracted reusable followup_item definition. - assets/shepherd-prompt.md: fixed verdict mapping bug (ship_with_followups + 0 blocking -> ready-to-merge, not needs-author-changes); added recommended_followups extraction step with required source_persona + optional fold_hint tagging. - assets/completion-prompt.md: full rewrite. Adds RECOMMENDED_FOLLOWUPS input; encodes FOLD vs DEFER classifier (FOLD: touches diff / single helper / regression trap / hermetic test / inline comment; DEFER: cross-cutting refactor / new feature / broad doc / architectural addition); per-FOLD item consultation with source_persona + python-architect lens; DEFER items filed as gh issue create tracking issues (never silently dropped); mid-flight reclassify rule to avoid stalls. - SKILL.md adds 'Bias toward folding recommended items' invariant and rewrites Phase 4 spawn contract (9 steps) to thread the recommended_followups channel end-to-end. Eval gate - +3 rubric anchors per content fixture (progress-diagram-header, mermaid-flowchart-rendered, dispatch-table-before-fanout) and +3 invariant anchors (recommended-followups-channel, fold-defer-classifier, tracking-issue-for-defer). - All 12 new anchors MATCH with_skill fixtures and MISS without_skill fixtures (clean value delta). - +3 no-fire trigger items for single-PR fold-in phrasing so the dispatcher will not misfire the batch outer-loop on single-PR fold work (e.g. 'fold the panel recommendations into PR #1234' remains apm-review-panel completion territory). Validation - Schema validates via jsonschema Draft7; accepts new shapes, rejects shepherd_return missing recommended_followups[]. - SKILL.md: 367 lines / 4483 tokens (caps: 500 / 5000). - Description: 965 / 1024 chars; mentions FOLD invariant. - 0 non-ASCII bytes across all modified files. - All 4 changed JSON files parse. Real-task evidence (this skill iteration was driven by a live run) - 5 of 6 in-flight community PRs had their panel recommendations folded in-PR by completion subagents following the new contract, yielding 22 folded items and 8 deferred-to-tracking items across PRs #1387, #1396, #1441, #1443, #1444. The 6th (#1442) is in flight as this lands. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * batch-bug-shepherd: add Phase 5 mergeability gate Adds a post-wave gate that re-probes mergeability for every PR the saga marked ready-to-merge, dispatches one conflict-resolution subagent per CONFLICTING PR, and partitions returns into four post-gate statuses before the final report claims anything is mergeable. Mergeability is post-wave truth, not pre-wave assumption: a PR that Phase 4 marked ready can stop being mergeable the moment the maintainer lands another PR onto main. Without this gate the report ships stale ready claims. Design driven through the genesis skill end-to-end (steps 1-6 handoff packet, steps 7a-7b coder pass, step 8 validation): - NEW Phase 5 (mergeability gate) between completion (Phase 4) and renamed final report (Phase 5 -> Phase 6). - Sub-phases 5a probe (read-only, single-thread, gh pr view --json mergeStateStatus), 5b fan-out (one conflict-resolution subagent per CONFLICTING PR), 5c trust-but-verify re-probe + four-way partition (resolved / requires-author-action / requires-human-judgment / resolution-failed). - NEW assets/conflict-resolution-prompt.md spawn body for 5b. Encodes rebase, faithful merge of both intents, mutation-break re-check, lint silent, --force-with-lease push, re-probe, resolution-confirmation comment. - NEW references/mergeability-gate.md load-on-demand orchestrator step-by-step (load trigger: WHEN ENTERING PHASE 5). Keeps SKILL.md under 5000-token budget. - Schema extends verdict-schema.json oneOf with conflict_resolution_return; --force-with-lease enforced via regex pattern guard on push_command field; bare --force rejected. Five rejection cases validated. - Two-comment-per-PR cap as new architecture invariant: at most one completion-confirmation (Phase 4) + one resolution-confirmation (Phase 5b) per PR. - Progress diagram extended with WAVE4 subgraph (P5a/P5b/P5c), skipped-state semantics, P5b dispatch table requirement. - Final report extended with three new partition sections plus a RESOLUTION CONFIRMATION COMMENT block and mergeability-gate disciplines line. - Evals: +3 content rubric anchors (mergeability-probe-cli, force-with-lease-on-push, post-wave-partition-columns) + 1 optional anchor (two-comment-cap); +1 fire + 2 no-fire trigger items; fixture diff shows the gate firing on a sweep with 2 conflicting PRs and the without-skill failure mode (stale ready claim). SKILL.md: 388 lines / 4867 tokens (budget 500/5000). ASCII only. CI lint pair silent. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * feat(bbs): add Phase 1.5 strategic-alignment gate + PRINCIPLES.md Adds a new wave between Phase 1 (triage) and Phase 2 (PR-in-flight cross-reference) that checks every LEGIT bug against the project's rejection contract before spending shepherd / fix / completion work on it. What changes: - NEW PRINCIPLES.md at repo root: 7 numbered principles encoding the project's hard nos (P1 no invented frontmatter; P2 multi-harness with traction gating; P3 vendor neutral; P4 UX floor is not a trade) plus 3 supporting principles (P5 portability; P6 reliability over magic; P7 community over feature count). Bound to apm-ceo + bbs Phase 1.5 + apm-triage-panel + apm-review-panel as the rejection contract. - NEW bbs Phase 1.5 strategic-alignment gate (WAVE 1.5): - one apm-ceo subagent per LEGIT row, in parallel - 4-state verdict: aligned | aligned-with-reservations | out-of-scope | wrong-direction - schema-validated returns; FAILS OPEN on infrastructure failure (malformed-x2 or non-citable principle) so legit bugs are never silently demoted under gate breakage - ABORTS only when apm-ceo.agent.md or PRINCIPLES.md itself is missing (operator-actionable error) - demoted rows flip to status triaged-deferred and SKIP Phase 2/3/4/5; surface in Phase 6 under 'Recommend close as out-of-scope' partition - aligned-with-reservations rows stay in saga; downstream phases surface the reservations in review prose - deferred-PR strategic-rejection comment subagent (S7+S4+A9) posts a courtesy comment on any open PR whose underlying issue was demoted, using the would-be Phase-4 completion-comment slot (two-comments-per-PR cap preserved) - Verdict schema extended with 5th oneOf member strategic_alignment_return (kind, issue, verdict, cited_principle, rationale, reservations). - Ground-truth table grows two columns (strategic_verdict + strategic_rationale) and one status value (triaged-deferred). - Progress diagram inserts P15 between P1 and P2; dispatch-table contract extends to Phase 1.5. - Final-report template adds 'Recommend close as out-of-scope' partition and 'Aligned with reservations' surfacing section. - 2 new fire trigger evals + 1 no-fire (PRINCIPLES.md authoring guard) + 1 new rubric anchor on the three-issues-mixed scenario. Genesis design artifact lives in the session plan store; SKILL.md body remains within 500-line / 5000-token budget (406 lines / 4943 tokens after trimming pre-existing verbose passages on operator-visibility, mergeability, fold-in, composition, and operating-contract sections to make room). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: danielmeppiel <danielmeppiel@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 51e9116 commit 9b4279e

19 files changed

Lines changed: 2119 additions & 251 deletions

PRINCIPLES.md

Lines changed: 105 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,105 @@
1+
# APM Principles
2+
3+
The hard contract APM is held to. Every PR, every release, every
4+
roadmap call cites these principles. The triage panel, the review
5+
panel, and the batch-bug-shepherd Phase 1.5 strategic-alignment
6+
gate cite them by number when accepting or rejecting work.
7+
8+
MANIFESTO = values. PRD = product pitch (currently framed at native
9+
platform owners). **This file = the rejection contract.** When a
10+
principle here conflicts with a feature request, the principle wins
11+
by default and shifts only via an explicit, written trade-off in
12+
the same PR.
13+
14+
## P1 -- No invented primitive frontmatter
15+
16+
APM emits to canonical schemas defined by upstream ecosystems:
17+
agentskills.io / Anthropic Claude Skills, GitHub Copilot, Cursor,
18+
Windsurf, Codex, Gemini, OpenCode. We do not invent `apm-*`
19+
frontmatter keys, top-level fields, or hidden attributes that
20+
downstream consumers must learn to honor APM-ness.
21+
22+
The primitive on disk must be readable, valid, and useful in the
23+
consuming harness with zero APM-specific tooling.
24+
25+
Rejection example: "Add an `apm-priority` frontmatter key so
26+
skills can self-rank in dispatch." NO. Dispatch ranking is the
27+
harness's problem, not a schema mutation.
28+
29+
## P2 -- Multi-harness with traction gating
30+
31+
APM ships to every harness with demonstrable user traction. Today:
32+
Copilot, Claude Code, Cursor, Windsurf, Codex, Gemini, OpenCode.
33+
Adding a new harness requires evidence: published download / install
34+
counts, named enterprise users, or a public ranking that places it
35+
in the top tier of agent runtimes.
36+
37+
We do not chase the long tail. A harness with zero documented users
38+
does not get a target adapter, period.
39+
40+
Rejection example: "Add a target for <obscure-harness>." NO unless
41+
there is a citable traction number.
42+
43+
## P3 -- Vendor neutral by construction
44+
45+
No primitive APM produces or installs may bake in a preferred LLM
46+
vendor, a preferred runtime, or a "works best with X" recommendation
47+
in shipped output. README, docs, and CLI output must remain
48+
runtime-agnostic where the surface is general.
49+
50+
Per-target adapters are allowed (they ARE the multi-harness promise);
51+
preferential framing inside neutral surfaces is not.
52+
53+
Rejection example: "Default `apm run` to invoke Claude when no
54+
runtime is configured." NO. Surface the missing config and require
55+
an explicit choice.
56+
57+
## P4 -- UX is the floor, not a trade
58+
59+
APM's adoption funnel runs through `apm init`, `apm install`, and
60+
`apm run`. No bug fix, hardening, security patch, or refactor lands
61+
if it makes those commands harder, slower, more verbose, or more
62+
confusing for a new user. The bug stays open until a UX-preserving
63+
fix exists.
64+
65+
This is asymmetric on purpose: a bug bites the affected user once;
66+
a bad install experience loses every future user silently.
67+
68+
Rejection example: "Fix #X by adding a required `--target` flag on
69+
`apm install`." NO. Find a fix that preserves target inference, or
70+
leave the issue open.
71+
72+
## P5 -- Portability over vendor lock-in
73+
74+
A primitive authored once must execute across every supported
75+
harness without modification. Lock-in of any flavor -- vendor,
76+
runtime, host -- is a regression.
77+
78+
## P6 -- Reliability over magic
79+
80+
Behavior must be predictable, auditable, and explainable in plain
81+
English. No silent normalization, no opaque heuristics, no "the
82+
agent decided." Every transformation has a name and a line in the
83+
changelog.
84+
85+
## P7 -- Community over feature count
86+
87+
External-contributor PRs and issues triage before internal
88+
nice-to-haves. A contributor lost is worse than a feature delayed.
89+
Surface every external interaction at the top of the queue.
90+
91+
## How this file is used
92+
93+
- `apm-ceo` cites by number in arbitration prose.
94+
- `batch-bug-shepherd` Phase 1.5 spawns one ceo subagent per
95+
triaged-LEGIT row, which returns a verdict + cited principle.
96+
- `apm-triage-panel` CEO arbiter cites a principle on every
97+
`decline-with-reason` rubric outcome.
98+
- `apm-review-panel` CEO synthesizer cites a principle when
99+
surfacing strategic implications in arbitration.
100+
101+
Any addition to this file requires the apm-ceo persona to ratify
102+
and ships in a PR that updates MANIFESTO.md cross-refs in the same
103+
commit. Removal of a principle is a breaking strategic change --
104+
requires CHANGELOG entry, migration line, and explicit `BREAKING:`
105+
prefix.

packages/batch-bug-shepherd/.apm/prompts/batch-bug-shepherd.prompt.md

Lines changed: 24 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -43,15 +43,36 @@ Targets for this run: **${input:targets}**
4343
plan.md). If `sweep-all` produced more than 20 candidates, ASK for
4444
confirmation; otherwise proceed.
4545

46+
Then RENDER the progress mermaid diagram per the skill's
47+
`assets/progress-diagram.md` -- every phase styled `pending`,
48+
with the candidate count `N` substituted into the P0/P1 labels.
49+
Print the live (empty) ground-truth table directly below it.
50+
This anchors the operator's view for the rest of the run.
51+
4652
4. INITIALIZE the ground-truth table in plan.md using the
4753
ground-truth-table asset shipped with the skill. One row per
4854
candidate. Status `pending-triage`.
4955

50-
5. EXECUTE the skill phases in order. For each phase boundary,
51-
reload the ground-truth table before spawning the next wave.
56+
5. EXECUTE the skill phases in order. For each phase boundary:
57+
- reload the ground-truth table from plan.md (B4 PLAN MEMENTO),
58+
- re-render the progress mermaid with the just-entered phase
59+
styled `active` and earlier phases `done` / `blocked` /
60+
`skipped` per the color contract,
61+
- print the live ground-truth table beneath the diagram,
62+
- and (for every fan-out wave: Phase 1, 3a, 3b, 4) print the
63+
dispatch table mapping each subagent_id to its target BEFORE
64+
spawning.
65+
66+
These renders are mandatory, not cosmetic -- they are the
67+
operator's only real-time signal that the saga is alive and
68+
what it's working on.
5269

5370
6. RENDER the final report from the final-report-template asset
54-
shipped with the skill at session end.
71+
shipped with the skill at session end. Use clickable GitHub
72+
issue / PR / author links so the operator can navigate without
73+
copy-paste. Re-render the progress mermaid one last time with
74+
every phase `done` (or `blocked` where the human-escalation
75+
queue is non-empty).
5576

5677
## Delegation
5778

0 commit comments

Comments
 (0)