Skip to content

Test agent detection and the process helpers - #47

Merged
nrodd merged 1 commit into
tests/harness-and-cifrom
tests/agent-helpers
Sep 28, 2026
Merged

nrodd merged 1 commit into
tests/harness-and-cifrom
tests/agent-helpers

Conversation

@nrodd

@nrodd nrodd commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Chunk 5 of the test plan. Two files — src/agents/helpers.test.ts (50 tests) and src/agents/index.test.ts (19). No source changes.

Based on #42 (the harness). GitHub will retarget this to main when that merges. Independent of #43–#46 — different files.

sanitizeTerminalOutput gets the most weight

Agents echo arbitrary repo content — READMEs, file bodies, command output — straight into the user's terminal through the wizard. Escape sequences in that stream can spoof output or drive the terminal, so what survives this function is a security boundary rather than a formatting choice.

  • Colour (SGR) is kept, so the agent's own output still renders
  • OSC 52 clipboard writes are stripped in both BEL and ST terminations — the wizard puts install prompts on the clipboard, so this one matters
  • Screen clear, cursor moves, scroll regions, device queries
  • C0 controls including carriage return, with tab and newline preserved
  • A table-driven test asserting the actual guarantee: no escape byte survives unless it opens a colour sequence

One thing worth knowing

ESC ( B (charset selection) leaves (B behind as plain text. The escape byte is removed, so the terminal has nothing to act on — the security property holds — but the payload characters aren't stripped. My first draft asserted 'text' and failed; the test now records what actually happens. Not proposing a change, just no longer a surprise.

runTerminalAgent's line buffer

Chunks split mid-line, blank lines delivered rather than swallowed, stdio shape per mode, close with a null code (signal kill) treated as failure, and a spawn error rejecting.

The one to keep: the flush on end for a final line that never got a trailing newline. That's where an agent's closing summary — and its last telemetry marker — would otherwise be dropped, and the funnel would silently lose the final step.

Detection

index.test.ts substitutes fakes for the four agent modules instead of probing the machine. The real AGENTS list hits PATH, ~/.claude, and macOS bundles, so results would depend on whatever the runner has installed.

  • A rejecting probe is dropped without costing the user the other agents
  • --agent <id> throws for something undetected, naming what was found, rather than silently falling back to a picker a CI run would hang on
  • Terminal agents are ordered before app handoffs, and the picker hints distinguish "runs automatically" from "opens the app"

A note on test hygiene

Two tests in my first draft were wrong in ways worth flagging, since both are easy to repeat:

  • A macAppPath test passed against /Applications/Cursor.app on my machine — that path is checked before the home directory, so a real install would win. Now uses a name nothing will have installed.
  • Control characters written as \uXXXX escapes ended up as literal invisible bytes in the file. They're built with String.fromCharCode now, and the file is verified to contain none.

Verification

69 tests passing (122 on this branch: chunk 0 plus this one), typecheck clean. Mutation-checked the two load-bearing behaviors — removing the end-of-stream flush and the .catch(() => null) on probes — and confirmed exactly the right tests fail, then reverted.


Note

Low Risk
Test-only additions with mocked I/O; no runtime behavior changes.

Overview
Adds Vitest coverage only — no production code changes — for agent process helpers and the agent catalog/selection flow (~69 new tests across two files).

helpers.test.ts exercises sanitizeTerminalOutput as a security boundary (SGR colour kept; OSC 52 clipboard writes, cursor/screen escapes, and C0 controls stripped; invariant that no stray ESC survives except SGR). It also mocks child_process/os to test which, path helpers, openAppAtDir, and runTerminalAgent (stdio modes, exit/signal handling, and stdout line buffering including flush on stream end for a final line without a newline).

index.test.ts replaces real agent modules with fakes so detectAgents / chooseAgent are deterministic: catalog ordering (terminal before app), resilient detection when a probe rejects, --agent strict errors for undetected IDs, interactive picker hints, and cancel handling.

Reviewed by Cursor Bugbot for commit 127f5fe. Bugbot is set up for automated code reviews on this repo. Configure here.

Chunk 5. Adds src/agents/helpers.test.ts (50 tests) and
src/agents/index.test.ts (19). No source changes.

sanitizeTerminalOutput gets the most attention: agents echo arbitrary
repo content into the user's terminal through the wizard, so what
survives is a security boundary. Colour is kept, OSC 52 clipboard writes
are stripped in both BEL and ST terminations, and a table-driven test
asserts the real guarantee — no escape byte survives unless it opens a
colour sequence.

One thing that surprised me and is now written down: ESC ( B leaves "(B"
behind as plain text. The escape byte is gone so the terminal can't act
on it, which is the property that matters, but the payload characters
are not removed.

runTerminalAgent's line buffer covers chunks split mid-line, blank lines
being delivered rather than swallowed, and the flush on 'end' for a
final line with no trailing newline — that last one is where an agent's
closing telemetry marker would otherwise be dropped.

index.ts substitutes fakes for the four agent modules rather than
probing the machine, so detection order and failure handling don't
depend on what the runner has installed. Covers a rejecting probe being
dropped without costing the user the other agents, and --agent throwing
for an undetected id rather than silently falling back to a picker a CI
run would hang on.

Control characters are built with String.fromCharCode so they stay
visible in the source rather than sitting in the file as invisible bytes.
@nrodd nrodd mentioned this pull request Sep 25, 2026
@nrodd
nrodd merged commit 2e7dece into tests/harness-and-ci Sep 28, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant