-
-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Description
Problem Statement
Modern agentic coding systems (Claude Code, GitHub Copilot Workspace, OpenCode, Cursor with Composer, etc.) build test-driven harness systems that run tests continuously during development loops. These agents:
- Make small, incremental changes to codebases
- Run tests after every change for validation
- Need fast feedback cycles to maintain productivity
- Often work in environments without git history (containers, temporary workspaces)
Running full test suites on every agent iteration creates a major bottleneck in agent productivity.
Proposed Solution
Add a --stale CLI option (inspired by Elixir Mix) that runs only test files whose transitive module dependencies have changed since the last --stale run.
How It Works
- First run: All tests execute, manifest generated with file mtimes
- Subsequent runs: Only tests with changed dependencies execute
- Tracking: Uses filesystem mtime comparison (fast, no git required)
- Dependencies: Tracks test → module → module's imports recursively
Example - Agent Workflow
# Agent modifies src/parser.ts
vitest run --stale
# ✓ Only runs tests importing parser.ts (directly/transitively)
# ✓ 10x faster than full suite
# ✓ Agent gets immediate feedback
# Agent makes another change
vitest run --stale
# ✓ Only newly affected tests run
# ✓ Agent maintains rapid iterationWhy Not --changed?
--changedrequires git (many agent environments don't have it)--changedcompares to a git ref (not optimized for rapid iteration)--staleis stateless and works anywhere--staleis faster (mtime check vs git diff)
Benefits for Agentic Systems
⚡ Speed: Only affected tests → 5-10x faster iteration
🎯 Precision: Transitive dependency tracking ensures no missed tests
🔄 Stateless: No git required → works in containers, CI, local
📊 Deterministic: Manifest-based → predictable test selection
Implementation
I have a complete, tested implementation ready in #9916:
- ✅ Mutually exclusive with
--changed - ✅ Honors
forceRerunTriggers - ✅ Watch mode compatible
- ✅ Integrates with Vite module graph
- ✅ 6 comprehensive tests, all passing
- ✅ Full documentation
- ✅ Zero breaking changes
- ✅ 20 files changed, +410 lines
- ✅ 7 atomic commits
Reference PR: #9916
Impact
This feature would make Vitest the first major JavaScript testing framework optimized for agentic coding workflows, positioning it at the forefront of the AI-assisted development revolution.
Questions for Maintainers
- Does this feature align with Vitest's roadmap?
- Any concerns about the manifest-based approach?
- Should this be a plugin instead? (I think core is better for consistency)
- Any naming preferences? (
--stale,--affected,--incremental?)
Alternatives considered:
- Using
--changed: Too git-dependent, not optimized for rapid iteration - Custom scripts: Every team would reinvent this, better as core feature
- Watch mode only: Doesn't work for agent batch workflows