Skip to content

fix(server): workspace search leaves out files Git ignores outside .gitignore - #35673

Open
juliusmarminge wants to merge 2 commits into
mainfrom
factory/issue-35587-a1794523647
Open

juliusmarminge wants to merge 2 commits into
mainfrom
factory/issue-35587-a1794523647

Conversation

@juliusmarminge

Copy link
Copy Markdown
Member

The native search index (fff-node 0.11.0, the latest stable) applies .gitignore files only: it does not read .git/info/exclude, core.excludesFile or the default global ignore file, and it reports those files with gitStatus 'clean', so its results carry no signal to filter on. WorkspaceSearchIndex.make now captures VcsProcess and loads Git's untracked ignored paths with git -c core.fsmonitor=false ls-files --others --ignored --exclude-standard --directory -z when the index is built and again on each refresh (refresh already runs after every agent run and file write). list, search (file, directory and mixed) and searchContents drop any entry whose path or a parent folder is in that set. A new searchVisible helper fetches a bigger page (x4 each time, up to the existing WORKSPACE_INDEX_PAGE_SIZE cap) when ignored entries fill the first page, so they cannot crowd out real matches; it is still one call when nothing is hidden. The result mappers now take a narrow {items,totalMatched} type. If git fails, times out or its output is cut off, the set is empty and behaviour matches main. No regex or parser was added; the ignored-path check walks a path's parent folders once each, so its cost grows with folder depth. Known gap: a file created between refreshes that a non-.gitignore rule would hide can show up until the next refresh.

Fixes #35587

Reproduction on main

Commit da345d2 (origin/main). 1) Ran the reporter's repro.sh unchanged against a fresh worktree after vp i: it exits 0, meaning the defect is present. In the 'configured' fixture git check-ignore reports gitignore-target.txt, info-exclude-target.txt and global-exclude-target.txt as ignored. WorkspaceEntries.search, flat list and searchContents all still return info-exclude-target.txt and global-exclude-target.txt; only gitignore-target.txt is excluded. The 'default' fixture (~/.config/git/ignore) shows the same for default-exclude-target.txt and info-exclude-target.txt. Directory-scoped list marks them all ignored:true, which is correct. Adding the same patterns to .gitignore (the 'workaround' fixture) excludes them. 2) Extended two existing tests in apps/server/src/workspace/WorkspaceEntries.test.ts. 'excludes gitignored paths for git repositories' gains files listed in .git/info/exclude and in a repository-local core.excludesFile. 'honors case sensitivity and gitignore rules' gains a file listed in .git/info/exclude. Both fail on main with expected [...] to not include 'info-excluded.txt' and an extra content match. Both runs called the production WorkspaceEntries service layer, not a dev server driven through the UI.

Evidence

Being captured: before and after in the real app. This section is filled in before the pull request can merge.

Tests

vp test run src/workspace/WorkspaceEntries.test.ts src/workspace/WorkspaceSearchIndex.test.ts: 31 passed (with WorkspaceSearchIndex.ts reverted to the repro base: 3 failed, the two repro cases plus the new page-growth case). vp run typecheck in apps/server: pass. vp run check:changed: pass (format, lint, knip, graphql, typecheck of 3 packages, tests of 3 files). Manual check: git ls-files --exclude-standard on a fixture lists the info/exclude, core.excludesFile and .gitignore entries.

Not in this change

  • Moving to an fff-node build that applies .git/info/exclude and global excludes natively would make this post-filter unnecessary; only 0.11.1 nightlies exist today.
  • Ignored files created between index refreshes can still show up until the next refresh; per-query git check-ignore would close that gap at the cost of one subprocess per keystroke.
  • Content search still scans ignored files inside fff and drops their hits afterwards, which uses part of its 250 ms time budget.

AI-assisted; verified with the tests and tools listed above.

…e.excludesFile

Failing tests for #35587: path search and content search return files Git
ignores through .git/info/exclude or core.excludesFile.

AI-assisted; verified with the tests and tools listed above.
…itignore

The native search index applies .gitignore files only, so files ignored
through .git/info/exclude, core.excludesFile or the default global ignore
file showed up in path search, the flat listing and content search.

The index now lists Git's ignored paths with
`git ls-files --others --ignored --exclude-standard --directory` when it
is built and on each refresh, and drops matching entries and grep hits.
Path searches widen their first page when hidden entries fill it.

AI-assisted; verified with the tests and tools listed above.
@github-actions github-actions Bot added vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. size:L 100-499 changed lines (additions + deletions). labels Nov 12, 2026
@macroscopeapp

macroscopeapp Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at da39114

Macroscope's review found this PR approvable — Workspace search now honors Git's global and repository-local exclusions as well as .gitignore. The change filters search results using bounded, read-only Git inspection without modifying files or persistent data.

You can add or adjust custom eligibility rules. Learn more.

@github-actions

Copy link
Copy Markdown
Contributor

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ The exact PR base did not have a successful artifact. Baseline uses the latest successful main measurement shown below.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 21.5 KiB 21.5 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 1 1 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.9 KiB 21.9 KiB 0 B (0.0%) 24.4 KiB ✅
Claude Live turn messages 1 1 0 (0.0%) 8 ✅

Other scenarios

All 64 scenario measurements match main.

All 64 scenario measurements
Provider Scenario Metric Main baseline This PR Impact PR ceiling
Codex short turn resume WebSocket decoded 10.5 KiB 10.5 KiB 0 B (0.0%) 12.7 KiB ✅
Codex short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn running, cold open HTTP decoded 281.7 KiB 281.7 KiB 0 B (0.0%) 302.7 KiB ✅
Codex long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Codex long turn running, history page HTTP decoded 92.4 KiB 92.4 KiB 0 B (0.0%) 107.4 KiB ✅
Codex long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Codex long turn finished, cold open HTTP decoded 381.5 KiB 381.5 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, socket cold open WebSocket decoded 381.6 KiB 381.6 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, work page HTTP decoded 186.2 KiB 186.2 KiB 0 B (0.0%) 200.2 KiB ✅
Codex long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Codex idle thread resume WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell snapshot HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Codex fleet shell subscribe WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Codex fleet shell resume check rows 0 0 0 0 ✅
Codex stale shell reload HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex stale shell reload rows 27 27 0 (0.0%) 27 ✅
Codex stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Codex stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex stale todo list cold open HTTP decoded 72.4 KiB 72.4 KiB 0 B (0.0%) 83.0 KiB ✅
Codex stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Codex stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅
Claude short turn resume WebSocket decoded 10.6 KiB 10.6 KiB 0 B (0.0%) 12.7 KiB ✅
Claude short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn running, cold open HTTP decoded 283.3 KiB 283.3 KiB 0 B (0.0%) 302.7 KiB ✅
Claude long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Claude long turn running, history page HTTP decoded 93.4 KiB 93.4 KiB 0 B (0.0%) 107.4 KiB ✅
Claude long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Claude long turn finished, cold open HTTP decoded 384.2 KiB 384.2 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, socket cold open WebSocket decoded 384.3 KiB 384.3 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, work page HTTP decoded 188.1 KiB 188.1 KiB 0 B (0.0%) 200.2 KiB ✅
Claude long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Claude idle thread resume WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell snapshot HTTP decoded 42.9 KiB 42.9 KiB 0 B (0.0%) 53.7 KiB ✅
Claude fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Claude fleet shell subscribe WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Claude fleet shell resume check rows 0 0 0 0 ✅
Claude stale shell reload HTTP decoded 42.9 KiB 42.9 KiB 0 B (0.0%) 53.7 KiB ✅
Claude stale shell reload rows 27 27 0 (0.0%) 27 ✅
Claude stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Claude stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude stale todo list cold open HTTP decoded 73.4 KiB 73.4 KiB 0 B (0.0%) 83.0 KiB ✅
Claude stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Claude stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅

Baseline: e253b88 · PR result: da39114 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@coderabbitai

coderabbitai Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The workspace search index now loads Git-ignored paths and filters them from listing, name-search, and content-search results. Name searches fetch additional results when ignored paths leave fewer visible results than requested.

Changes: Workspace search

Layer / File(s) Summary
Ignored-path discovery and refresh
apps/server/src/workspace/WorkspaceSearchIndex.ts
The index runs git ls-files to load ignored paths, normalizes paths for descendant checks, and reloads the set during refresh.
Search filtering and validation
apps/server/src/workspace/WorkspaceSearchIndex.ts, apps/server/src/workspace/WorkspaceEntries.test.ts, apps/server/src/workspace/WorkspaceSearchIndex.test.ts
Listing, name-search, and content-search results skip ignored paths. Name searches fetch more results when needed and adjust totalMatched. Tests cover Git ignore sources and a limited search result page.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related PRs

  • pingdotgg/t3code#1000: Applies Git-derived exclusions to workspace path and content search through a different mechanism.
  • pingdotgg/t3code#34831: Adds gitignored nested worktree files to exact-path search results, which overlaps with this PR’s ignored-path filtering.

Suggested labels: bugfix

Suggested reviewers: t3dotgg, macroscopeapp

Merge Risk: Minimal · up to da391

Workspace search now filters files ignored through Git's exclude sources. If Git cannot supply the ignored paths, search behaves as it did before. No merge-blocking risk was identified.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title describes the main change: workspace search omitted files that Git ignores through sources other than .gitignore. The changeset adds ignored-path loading from .git/info/exclude, core.exclude…
Description check Passed The description follows the repository template closely. It covers the problem (fff-node applies only .gitignore), the change (ignored-path loading via VcsProcess, filtering in list/search/searchConte…
Linked Issues check Passed [#35587] Requirements covered by the diff at head:
1. Path search, flat listing, and content search exclude untracked paths ignored through .git/info/exclude, core.excludesFile, and the default global…
Out of Scope Changes check Passed All changed hunks connect to the objectives of #35587. The import of VcsProcess, the ignored-path loader, isIgnoredPath, mixedItemPath, and searchVisible implement the ignore filtering. The narrowed S…
Full details: Title check

Explanation

The title describes the main change: workspace search omitted files that Git ignores through sources other than .gitignore. The changeset adds ignored-path loading from .git/info/exclude, core.excludesFile, and the global ignore file, then filters those paths from search results. The title uses the conventional commit format and is specific enough for a teammate to understand the primary change.

Full details: Description check

Explanation

The description follows the repository template closely. It covers the problem (fff-node applies only .gitignore), the change (ignored-path loading via VcsProcess, filtering in list/search/searchContents, page growth via searchVisible, and fallback to empty set on git failure), and a linked issue (#35587). It gives concrete verification steps with observed results on main and after the change. The Scope section does not explicitly state why the change is a bug fix rather than a feature, though the linked issue documents the defect. The Evidence section is explicitly marked as pending, and the template asks for before/after evidence only for UI changes, which this PR does not contain. The description also includes a Not in this change section, which is extra but not disallowed.

Full details: Linked Issues check

Explanation

[#35587] Requirements covered by the diff at head:

  1. Path search, flat listing, and content search exclude untracked paths ignored through .git/info/exclude, core.excludesFile, and the default global ignore file. WorkspaceSearchIndex.make runs git ls-files --others --ignored --exclude-standard --directory -z at build and on refresh. list, fileSearch, directorySearch, mixedSearch (via searchVisible) and searchContents drop matching paths and their descendants (isIgnoredPath walks parent folders).
  2. Preserve existing behavior: --others limits the set to untracked paths, so staged and tracked files are not hidden. .gitignore-matched files are already excluded by fff and stay excluded. The directory-scoped listing path is not changed by this diff, so it keeps ignored entries marked ignored: true.
  3. Page growth: searchVisible refetches with a larger page, capped at WORKSPACE_INDEX_PAGE_SIZE, when hidden entries fill the first page.
    Tests: WorkspaceEntries.test.ts adds .git/info/exclude and core.excludesFile cases for path search and a .git/info/exclude case for content search. WorkspaceSearchIndex.test.ts adds a page-growth case. The PR reports these tests fail on main. Limits: the known gap (files created between refreshes) is a documented limitation, not a stated requirement of the issue.
Full details: Out of Scope Changes check

Explanation

All changed hunks connect to the objectives of #35587. The import of VcsProcess, the ignored-path loader, isIgnoredPath, mixedItemPath, and searchVisible implement the ignore filtering. The narrowed SearchPage<I> type replaces fff result types so the mappers accept filtered results. That change supports the implementation and does not change behavior. The test changes cover the new ignore sources and page growth. No unrelated behavior change appears in the diff. The PR description's notes about performance cost and content-search scanning are disclosed trade-offs, not unrelated changes.

Finishing Touches

Generate unit tests (beta)

  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Recent review info

Run configuration

Commits

Reviewing files that changed from the base of the PR and between c4e74fe and da39114.

Files selected for processing: 3

  • apps/server/src/workspace/WorkspaceEntries.test.ts
  • apps/server/src/workspace/WorkspaceSearchIndex.test.ts
  • apps/server/src/workspace/WorkspaceSearchIndex.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 2 remain after this review.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L 100-499 changed lines (additions + deletions). vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Workspace search returns files excluded by .git/info/exclude and global Git ignore files

1 participant