Commit 9b4279e
feat(batch-bug-shepherd): operator visibility + fold-in invariant (#1451)
* feat(batch-bug-shepherd): operator visibility + fold-in invariant
Refactors the batch-bug-shepherd skill to address two genesis-validated
blockers and ship two missing capabilities discovered during a real
sweep-all run over 14 microsoft/apm bugs:
1. OPERATOR VISIBILITY (was: silent 30-minute fan-outs)
- New asset assets/progress-diagram.md: mermaid template + 5-state
palette (pending/active/done/blocked/skipped) + per-phase render
rules + dispatch-table contract.
- SKILL.md adds 'Operator visibility is a contract' invariant; each
phase boundary re-renders the diagram with current-phase coloring
and prints a subagent_id -> target dispatch table BEFORE fan-out.
- Operator can follow long sagas at a glance instead of waiting in
the dark for the next checkpoint.
2. FOLD-IN INVARIANT (was: panel recommendations silently dropped)
- assets/verdict-schema.json: shepherd_return gains required
recommended_followups[] channel; completion_return gains
folded_followups[] + deferred_followups[]; extracted reusable
followup_item definition.
- assets/shepherd-prompt.md: fixed verdict mapping bug
(ship_with_followups + 0 blocking -> ready-to-merge, not
needs-author-changes); added recommended_followups extraction
step with required source_persona + optional fold_hint tagging.
- assets/completion-prompt.md: full rewrite. Adds
RECOMMENDED_FOLLOWUPS input; encodes FOLD vs DEFER classifier
(FOLD: touches diff / single helper / regression trap / hermetic
test / inline comment; DEFER: cross-cutting refactor / new
feature / broad doc / architectural addition); per-FOLD item
consultation with source_persona + python-architect lens;
DEFER items filed as gh issue create tracking issues (never
silently dropped); mid-flight reclassify rule to avoid stalls.
- SKILL.md adds 'Bias toward folding recommended items' invariant
and rewrites Phase 4 spawn contract (9 steps) to thread the
recommended_followups channel end-to-end.
Eval gate
- +3 rubric anchors per content fixture
(progress-diagram-header, mermaid-flowchart-rendered,
dispatch-table-before-fanout) and +3 invariant anchors
(recommended-followups-channel, fold-defer-classifier,
tracking-issue-for-defer).
- All 12 new anchors MATCH with_skill fixtures and MISS
without_skill fixtures (clean value delta).
- +3 no-fire trigger items for single-PR fold-in phrasing so the
dispatcher will not misfire the batch outer-loop on single-PR
fold work (e.g. 'fold the panel recommendations into PR #1234'
remains apm-review-panel completion territory).
Validation
- Schema validates via jsonschema Draft7; accepts new shapes,
rejects shepherd_return missing recommended_followups[].
- SKILL.md: 367 lines / 4483 tokens (caps: 500 / 5000).
- Description: 965 / 1024 chars; mentions FOLD invariant.
- 0 non-ASCII bytes across all modified files.
- All 4 changed JSON files parse.
Real-task evidence (this skill iteration was driven by a live run)
- 5 of 6 in-flight community PRs had their panel recommendations
folded in-PR by completion subagents following the new contract,
yielding 22 folded items and 8 deferred-to-tracking items across
PRs #1387, #1396, #1441, #1443, #1444. The 6th (#1442) is in
flight as this lands.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* batch-bug-shepherd: add Phase 5 mergeability gate
Adds a post-wave gate that re-probes mergeability for every PR the
saga marked ready-to-merge, dispatches one conflict-resolution
subagent per CONFLICTING PR, and partitions returns into four
post-gate statuses before the final report claims anything is
mergeable.
Mergeability is post-wave truth, not pre-wave assumption: a PR
that Phase 4 marked ready can stop being mergeable the moment the
maintainer lands another PR onto main. Without this gate the
report ships stale ready claims.
Design driven through the genesis skill end-to-end (steps 1-6
handoff packet, steps 7a-7b coder pass, step 8 validation):
- NEW Phase 5 (mergeability gate) between completion (Phase 4) and
renamed final report (Phase 5 -> Phase 6).
- Sub-phases 5a probe (read-only, single-thread, gh pr view --json
mergeStateStatus), 5b fan-out (one conflict-resolution subagent
per CONFLICTING PR), 5c trust-but-verify re-probe + four-way
partition (resolved / requires-author-action /
requires-human-judgment / resolution-failed).
- NEW assets/conflict-resolution-prompt.md spawn body for 5b.
Encodes rebase, faithful merge of both intents, mutation-break
re-check, lint silent, --force-with-lease push, re-probe,
resolution-confirmation comment.
- NEW references/mergeability-gate.md load-on-demand orchestrator
step-by-step (load trigger: WHEN ENTERING PHASE 5). Keeps
SKILL.md under 5000-token budget.
- Schema extends verdict-schema.json oneOf with
conflict_resolution_return; --force-with-lease enforced via
regex pattern guard on push_command field; bare --force
rejected. Five rejection cases validated.
- Two-comment-per-PR cap as new architecture invariant: at most
one completion-confirmation (Phase 4) + one
resolution-confirmation (Phase 5b) per PR.
- Progress diagram extended with WAVE4 subgraph (P5a/P5b/P5c),
skipped-state semantics, P5b dispatch table requirement.
- Final report extended with three new partition sections plus a
RESOLUTION CONFIRMATION COMMENT block and mergeability-gate
disciplines line.
- Evals: +3 content rubric anchors (mergeability-probe-cli,
force-with-lease-on-push, post-wave-partition-columns) + 1
optional anchor (two-comment-cap); +1 fire + 2 no-fire trigger
items; fixture diff shows the gate firing on a sweep with 2
conflicting PRs and the without-skill failure mode (stale ready
claim).
SKILL.md: 388 lines / 4867 tokens (budget 500/5000). ASCII only.
CI lint pair silent.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* feat(bbs): add Phase 1.5 strategic-alignment gate + PRINCIPLES.md
Adds a new wave between Phase 1 (triage) and Phase 2 (PR-in-flight
cross-reference) that checks every LEGIT bug against the project's
rejection contract before spending shepherd / fix / completion work
on it.
What changes:
- NEW PRINCIPLES.md at repo root: 7 numbered principles encoding
the project's hard nos (P1 no invented frontmatter; P2 multi-harness
with traction gating; P3 vendor neutral; P4 UX floor is not a
trade) plus 3 supporting principles (P5 portability; P6 reliability
over magic; P7 community over feature count). Bound to apm-ceo +
bbs Phase 1.5 + apm-triage-panel + apm-review-panel as the
rejection contract.
- NEW bbs Phase 1.5 strategic-alignment gate (WAVE 1.5):
- one apm-ceo subagent per LEGIT row, in parallel
- 4-state verdict: aligned | aligned-with-reservations |
out-of-scope | wrong-direction
- schema-validated returns; FAILS OPEN on infrastructure failure
(malformed-x2 or non-citable principle) so legit bugs are
never silently demoted under gate breakage
- ABORTS only when apm-ceo.agent.md or PRINCIPLES.md itself is
missing (operator-actionable error)
- demoted rows flip to status triaged-deferred and SKIP Phase
2/3/4/5; surface in Phase 6 under 'Recommend close as
out-of-scope' partition
- aligned-with-reservations rows stay in saga; downstream phases
surface the reservations in review prose
- deferred-PR strategic-rejection comment subagent (S7+S4+A9)
posts a courtesy comment on any open PR whose underlying issue
was demoted, using the would-be Phase-4 completion-comment
slot (two-comments-per-PR cap preserved)
- Verdict schema extended with 5th oneOf member
strategic_alignment_return (kind, issue, verdict, cited_principle,
rationale, reservations).
- Ground-truth table grows two columns (strategic_verdict +
strategic_rationale) and one status value (triaged-deferred).
- Progress diagram inserts P15 between P1 and P2; dispatch-table
contract extends to Phase 1.5.
- Final-report template adds 'Recommend close as out-of-scope'
partition and 'Aligned with reservations' surfacing section.
- 2 new fire trigger evals + 1 no-fire (PRINCIPLES.md authoring
guard) + 1 new rubric anchor on the three-issues-mixed scenario.
Genesis design artifact lives in the session plan store; SKILL.md
body remains within 500-line / 5000-token budget (406 lines /
4943 tokens after trimming pre-existing verbose passages on
operator-visibility, mergeability, fold-in, composition, and
operating-contract sections to make room).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: danielmeppiel <danielmeppiel@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent 51e9116 commit 9b4279e
19 files changed
Lines changed: 2119 additions & 251 deletions
File tree
- packages/batch-bug-shepherd/.apm
- prompts
- skills/batch-bug-shepherd
- assets
- evals
- content
- fixtures
- references
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
Lines changed: 24 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
46 | 52 | | |
47 | 53 | | |
48 | 54 | | |
49 | 55 | | |
50 | | - | |
51 | | - | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
52 | 69 | | |
53 | 70 | | |
54 | | - | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
55 | 76 | | |
56 | 77 | | |
57 | 78 | | |
| |||
0 commit comments