Repository navigation
Test the integrations catalog and picker - #46
Merged
Merged
Conversation
Chunk 4. Adds src/integrations.test.ts, 75 tests. No source changes. The picker only steers the agent — whatever it misses, the agent's own Step 2 explore still finds — so the weight is on the catalog's internal consistency and the --integrations parsing that CI runs depend on. Catalog invariants run per entry, so a failure names the tool: unique ids and labels, ids lowercase and spaceless (selectIntegrations lowercases before matching, so a capitalised id would be unreachable from the flag), every tool detectable by a package or a window global, every linkage example naming the session URL, and no npm package claimed by two tools. detectInstalledIntegrations runs against real temp directories, since returning [] rather than throwing on a missing or unparseable package.json is the property that matters. The interactive flow covers the collapse: a detection confirms the detected set behind one "any others?" instead of showing all fifteen options, and saying yes opens the picker with those pre-checked and hinted but still uncheckable. Cancels at all three prompts raise CancelledError. One recorded behavior: a repeated id in --integrations is not deduped, so the tool is listed twice in the prompt.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Chunk 4 of the test plan.
src/integrations.test.ts, 75 tests. No source changes.Where the weight went
The picker only steers the agent — anything it misses, the agent's own Step 2 explore still detects. So this doesn't chase prompt coverage. It leans on two things that can actually break a run:
Catalog consistency. The catalog is hand-maintained and feeds three places at once: the picker, package detection, and the linkage examples pasted into the agent prompt. These run per-entry so a failure names the tool:
selectIntegrationslowercases before matching, so a capitalised id in the catalog would be silently unreachable from--integrationslinkageExamplenon-empty and actually naming the session URL--integrationsparsing, which is the path CI and scripted runs take: case-insensitive matching, trimming, unknown names routed tootherwith their original case preserved, empty segments from a doubled comma dropped, and user order preserved rather than catalog order.The rest
detectInstalledIntegrationsagainst real temp directories — deps and devDeps, package aliases, several tools at once in catalog order, and the four ways it must return[]rather than throw (no file, unparseable file, missing directory, no deps).detected— but still uncheckable, since a pre-check is a default and not a lock.CancelledError. This is the one place in the flow where a cancel still aborts the run, which is worth having pinned before chunk 7, where the same gesture must not abort.One recorded behavior
A repeated id in
--integrationsisn't deduped, so--integrations posthog,posthoglists PostHog twice in the agent prompt. Harmless, and deduping is a behavior change rather than a test, so there's just a test saying so.Verification
75 tests passing (128 on this branch: chunk 0 plus this one),
typecheckclean. Mutation-checked two assertions — dropping the.toLowerCase()in flag matching, and returning an empty set instead of the detected one — and confirmed exactly those tests fail, then reverted.Note
Low Risk
Test-only PR with no runtime or CLI behavior changes.
Overview
Adds
src/integrations.test.ts(~75 Vitest cases) with no production code changes. Tests pin behavior the CLI and agent rely on: hand-maintainedINTEGRATIONSinvariants (unique ids/labels, lowercase ids, detectability, linkage examples, no duplicate npm packages),detectInstalledIntegrationsagainst temppackage.jsontrees (including safe empty results on bad/missing input), andselectIntegrationsfor both--integrations(case/trim, order,other, no prompts) and interactive flows (full picker vs collapsed “any others?” when deps are detected,__other__free text, catalog ordering,CancelledErroron cancel).Documents current behavior that repeated flag ids are not deduped.
Reviewed by Cursor Bugbot for commit a4adf66. Bugbot is set up for automated code reviews on this repo. Configure here.