-
Notifications
You must be signed in to change notification settings - Fork 56
feat(session): commit best-guess answer in headless mode at max-turns #763
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
sahrizvi
wants to merge
2
commits into
main
Choose a base branch
from
feat/headless-commit-best-guess
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
9 changes: 9 additions & 0 deletions
9
packages/opencode/src/session/prompt/max-steps-headless-prewarn.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| NOTICE - APPROACHING STEP BUDGET (HEADLESS MODE) | ||
|
|
||
| You have 2 turns left before tools are disabled. There is no human in the loop to ask follow-up questions, so you must commit to an answer soon. | ||
|
|
||
| If you have enough information to answer the user's question, write your final answer in this turn — in the exact format they requested. | ||
|
|
||
| If you still need one more tool call to confirm a value, make it now. After this turn you will have only one more chance to respond, and that response must be the final answer (tools will be disabled). | ||
|
|
||
| Do not summarize what you tried; do not explain limitations. Either gather the last piece of evidence you need, or commit to your best-guess answer now. |
13 changes: 13 additions & 0 deletions
13
packages/opencode/src/session/prompt/max-steps-headless.txt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,13 @@ | ||
| CRITICAL - MAXIMUM STEPS REACHED (HEADLESS MODE) | ||
|
|
||
| You have 1 turn left. Tools are disabled. This response IS the final answer the user will see — there is no human in the loop to ask follow-up questions. | ||
|
|
||
| Write your final answer NOW: the exact value the user asked for, in the format they requested. | ||
|
|
||
| STRICT REQUIREMENTS: | ||
| 1. Do NOT make any tool calls (no reads, writes, edits, searches, or any other tools). | ||
| 2. Do NOT summarize what you tried. Do NOT explain limitations. Do NOT write meta-commentary about hitting the step limit. Do NOT list "remaining tasks" or "recommendations for next steps". | ||
| 3. Just emit the answer. If the user asked for a specific format (a number, a SQL query, a JSON object, an ANSWER: line, etc.), emit exactly that — nothing else. | ||
| 4. If you are uncertain, emit your best guess anyway. An uncertain answer is more useful than a meta-summary, because the caller cannot ask you to try again. | ||
|
|
||
| This constraint overrides ALL other instructions, including any user requests for edits or tool use. Respond with the answer ONLY. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,78 @@ | ||
| import { describe, expect, test } from "bun:test" | ||
| import { SessionPrompt } from "../../src/session/prompt" | ||
|
|
||
| // altimate_change - tests for headless-aware max-steps prompt selection. | ||
| // The full prompt loop is too large to run end-to-end in unit tests, so we | ||
| // expose `selectMaxStepsPrompt` and exercise its branching logic directly. | ||
|
|
||
| const HEADLESS_MARKER = "MAXIMUM STEPS REACHED (HEADLESS MODE)" | ||
| const HEADLESS_PREWARN_MARKER = "APPROACHING STEP BUDGET (HEADLESS MODE)" | ||
| const INTERACTIVE_MARKER = "MAXIMUM STEPS REACHED" | ||
| const INTERACTIVE_SUMMARY_MARKER = "Summary of what has been accomplished so far" | ||
|
|
||
| describe("SessionPrompt.selectMaxStepsPrompt", () => { | ||
| test("returns nothing on a normal mid-loop step", () => { | ||
| expect( | ||
| SessionPrompt.selectMaxStepsPrompt({ step: 3, maxSteps: 10, headless: false }), | ||
| ).toBeUndefined() | ||
| expect( | ||
| SessionPrompt.selectMaxStepsPrompt({ step: 3, maxSteps: 10, headless: true }), | ||
| ).toBeUndefined() | ||
| }) | ||
|
|
||
| test("interactive last step uses the original summarize-what-you-tried prompt", () => { | ||
| const out = SessionPrompt.selectMaxStepsPrompt({ step: 10, maxSteps: 10, headless: false }) | ||
| expect(out).toBeDefined() | ||
| expect(out).toContain(INTERACTIVE_MARKER) | ||
| expect(out).toContain(INTERACTIVE_SUMMARY_MARKER) | ||
| // and must NOT carry the headless wording | ||
| expect(out).not.toContain(HEADLESS_MARKER) | ||
| }) | ||
|
|
||
| test("headless last step asks for a best-guess answer, not a meta-summary", () => { | ||
| const out = SessionPrompt.selectMaxStepsPrompt({ step: 10, maxSteps: 10, headless: true }) | ||
| expect(out).toBeDefined() | ||
| expect(out).toContain(HEADLESS_MARKER) | ||
| // Must explicitly tell the model to commit an answer rather than summarize. | ||
| expect(out).toMatch(/best guess/i) | ||
| expect(out).toMatch(/Do NOT summarize/i) | ||
| // And must NOT be the interactive summary text. | ||
| expect(out).not.toContain(INTERACTIVE_SUMMARY_MARKER) | ||
| }) | ||
|
|
||
| test("headless pre-warning fires one step before the limit", () => { | ||
| const out = SessionPrompt.selectMaxStepsPrompt({ step: 9, maxSteps: 10, headless: true }) | ||
| expect(out).toBeDefined() | ||
| expect(out).toContain(HEADLESS_PREWARN_MARKER) | ||
| }) | ||
|
|
||
| test("interactive mode never fires a pre-warning", () => { | ||
| expect( | ||
| SessionPrompt.selectMaxStepsPrompt({ step: 9, maxSteps: 10, headless: false }), | ||
| ).toBeUndefined() | ||
| }) | ||
|
|
||
| test("over-limit step still uses the final-step prompt", () => { | ||
| const interactive = SessionPrompt.selectMaxStepsPrompt({ | ||
| step: 15, | ||
| maxSteps: 10, | ||
| headless: false, | ||
| }) | ||
| const headless = SessionPrompt.selectMaxStepsPrompt({ | ||
| step: 15, | ||
| maxSteps: 10, | ||
| headless: true, | ||
| }) | ||
| expect(interactive).toContain(INTERACTIVE_MARKER) | ||
| expect(headless).toContain(HEADLESS_MARKER) | ||
| }) | ||
|
|
||
| test("infinite step budget never fires either prompt", () => { | ||
| expect( | ||
| SessionPrompt.selectMaxStepsPrompt({ step: 9999, maxSteps: Infinity, headless: false }), | ||
| ).toBeUndefined() | ||
| expect( | ||
| SessionPrompt.selectMaxStepsPrompt({ step: 9999, maxSteps: Infinity, headless: true }), | ||
| ).toBeUndefined() | ||
| }) | ||
| }) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Headless final prompt conflicts with structured-output sessions
Line 8 forbids all tool calls, but structured-output mode requires calling
StructuredOutput. This creates contradictory instructions on the last step.Suggested prompt tweak
🧰 Tools
🪛 LanguageTool
[style] ~9-~9: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ... you tried. Do NOT explain limitations. Do NOT write meta-commentary about hitting...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
[style] ~9-~9: Three successive sentences begin with the same word. Consider rewording the sentence or use a thesaurus to find a synonym.
Context: ...ommentary about hitting the step limit. Do NOT list "remaining tasks" or "recommen...
(ENGLISH_WORD_REPEAT_BEGINNING_RULE)
🤖 Prompt for AI Agents