Skip to content

Reduce repeated sub-agent work with explicit response forms - #565

Open
caihongwei2006 wants to merge 3 commits into
scitix:mainfrom
caihongwei2006:codex/subagent-response-forms
Open

Reduce repeated sub-agent work with explicit response forms#565
caihongwei2006 wants to merge 3 commits into
scitix:mainfrom
caihongwei2006:codex/subagent-response-forms

Conversation

@caihongwei2006

Copy link
Copy Markdown

Summary

Free-form sub-agent summaries can omit requested facts and force parent agents to repeat delegated work. Add parent-defined response forms so each child returns directly usable answers, reducing the need for follow-up delegation or an extra synthesis run.

Solution

  • Support fill-in and single-choice questions using plain-text name:\nanswer fields; expand choice keys to their meanings before parent delivery.
  • Preserve complete filled answers across single, batch, synthesis, and background paths instead of clipping them into report capsules. Background batches without synthesis now deliver item answers rather than only a completion count.
  • Mark missing, duplicate, or invalid answers explicitly, preserve valid fields, and do not launch an automatic repair run. Retain questions and raw answers for audit.

Compatibility

Breaking tool contract: new spawn_subagent calls must include response_form. Update manually authored calls and integrations to supply at least one question. The result envelope is unchanged; summary and reduce_summary contain the filled plain-text answers. Model context limits still apply.

Test Plan

  • Parser and tool contract tests, including choice expansion, multiline evidence, invalid answers, and long responses.
  • Runtime tests for foreground single/batch/synthesis and background single/batch delivery, including audit persistence and no automatic repair child.
  • npm test: 303 files passed; 6,749 tests passed, 2 skipped.
  • npm run build (TypeScript).
  • Live workload latency and delegation-count comparison after deployment.

jtang added 3 commits September 7, 2026 13:22
Return complete, parent-defined answers so delegated findings can be used without repeat investigation. Expand choice keys to text and deliver item answers for background batches without synthesis.

BREAKING CHANGE: spawn_subagent calls must supply response_form.

@LikiosSedo LikiosSedo left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking this on — a parent-owned answer contract is the right direction, and several pieces here are solid (fail-fast form validation before any child starts, no automatic repair run, option-key expansion, item answers in background notifications instead of a completion count). Requesting changes on three points, plus a note on the diagnosis.

On the diagnosis

The PR frames the problem as "free-form summaries omit requested facts, so the parent repeats work". In the production case that motivated this work (session ca45ade7, 2026-09-03) the children did return the five-section Findings Report the parent's template asked for — the format was not free-form. What went wrong was content: the parent template dropped the alert object (claude-fable-5) and the children matched keywords on another model's Novita 429s. A cause: A/B form does not stop a child from answering the right shape about the wrong target. That fix is anchor injection (brief.target from the caller contract, SiCore MR !1119) into the child prompt. Structured returns are still worth having, but that changes how hard we should push them.

For reference, none of Claude Code, Codex or grok-build require a response form on general-purpose spawns: all three send free-form prompts and return the child's final text verbatim; structured contracts exist only on specific high-value paths (Claude Code Workflow schema, Codex guardian/review/CSV jobs, grok-build's /goal verifier), and every parser there keeps a lenient tier before failing. grok-build once shipped a per-spawn contract parameter (persona) and later removed it.

Must change

1. response_form is required on every call (spawn-subagent.ts, required includes it). This breaks every existing caller (prompt.ts fan-out guidance, skills, tests) and forces the parent to design a questionnaire even for "go look at X and report back". Please make it optional — or require it only when it is actually needed for machine merging (items.length > 1 or reduce_prompt present).

2. A malformed form flips the child to status: "failed" (session.ts, if (formResult && !formResult.valid && status === "done") status = "failed"). The parser rejects 原因: A on one line (the most common model output — and the test pins it as invalid), any introductory sentence before the first heading, A. / A: / A), bold headings. Those are correct answers written differently. Once the child is "failed", the new tool description forbids the parent from re-dispatching or verifying, so the parent has to do the work itself — exactly the repeated work this PR sets out to remove. Suggest:

  • lenient parsing tiers (accept name: answer on one line, strip preamble/fences, accept A/A./A:/A)/lowercase), in the spirit of Codex's "strict → brace-slice → fail";
  • report format problems as status: "partial" with a separate form_errors[] in the envelope, keeping summary as the raw answer. failed should mean the investigation failed, not that a heading was on the wrong line. Fail-closed is right for safety verdicts (guardian-style), not for SRE evidence gathering.

3. Tool description prose (~1,500 chars added to the schema description, paid by every agent on every turn). Parts of it contradict each other ("treat the delegated attempt as complete … A completed attempt does not mean its conclusion is correct" vs "Do NOT repeat its trajectory for ANY confirmation purpose"), and "lead to a conclusion using ONE dispatch … Trust your subagents" turns an engineering goal into an order to the model. In ca45ade7 the parent's own re-check is what pulled the conclusion back from Novita to Bedrock; a blanket "never verify" is a dangerous default for SRE. Please keep the description to the contract itself (fields, format, one example) and move posture text to the general-purpose systemPromptAddendum, worded like Codex does it: "do not redo the child's trajectory to confirm it; when a conclusion drives an action, verify with an independent, cheaper check".

Should change

  • Reduce input is unbounded (buildReduceInput(..., Number.POSITIVE_INFINITY), and the reduce summary skips truncation). N items × long evidence fields go straight into the reduce prompt; "normal model context limits still apply" means it fails at the model. Keep a generous budget with an annotated cut, or spill to a file (Codex CSV jobs and grok-build both go to files past a threshold).
  • Reduce shares the item form. Reduce questions are usually group-level ("group causes into network/storage/other") and don't fit the per-item cause: A/B shape — the design doc's own example shows the mismatch. Allow a separate reduce_form.
  • No anchor check. Nothing in the form lets the runtime tell whether the child answered about the intended target. A runtime-implicit target_echo field (child restates the entity it investigated), compared against chat.send.params.brief.target, would catch the ca45ade7 failure class for one line of code.
  • Tests cover ideal output only. Please add table-driven cases from real transcripts (the ca45ade7 children's Findings Reports are a good fixture): one-line answers, bold headings, trailing sentences.
  • Minor: auditPrompt records the full prompt only when a form is present (two different audit shapes); option keys are restricted to ^[A-Z][A-Z0-9_]*$ while the docs/tests use Chinese field names — consider allowing the same for keys; lone \r is not normalized.

Suggest splitting out

The background-batch notification change (deliver item answers instead of a completion count, task-notification.ts) is independently valuable and non-breaking — it could land on its own while the form contract is reworked.

Interaction with the caller contract

The two are complementary: the contract governs what goes in (the parent gets a brief with target), this PR governs what comes out. Both touch buildSpawnedSubagentPrompt; when merging, the anchor block should precede Task: and the response form should stay last.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants