Skip to content

Commit 981c13f

Browse files
continue[bot]Continuepeter-parker
committed
test: add comprehensive test coverage for MCP allowHeadless feature
- Add edge case tests for disconnected servers, empty connections, and empty tools - Add permission policy interaction tests with wildcards and argumentMatches - Add integration tests for mixed built-in and MCP tools - Add test for built-in tools behavior in headless mode - Add tool ordering and mode transition tests Improves test coverage for PR #9328 with 12 new test cases covering: - Error handling and graceful degradation - Complex permission policy scenarios - Real-world integration scenarios - Security boundary validation Generated with [Continue](https://continue.dev) Co-Authored-By: Continue <noreply@continue.dev> Co-authored-by: peter-parker <e2e@continue.dev>
1 parent 2cfb204 commit 981c13f

4 files changed

Lines changed: 593 additions & 45 deletions

File tree

TEST_IMPROVEMENTS.md

Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
# Test Improvements for MCP Headless Mode (PR #9328)
2+
3+
## Summary
4+
5+
This PR adds comprehensive test coverage for the `allowHeadless` feature introduced in PR #9328. The new tests ensure robust behavior of MCP tools in headless mode across various scenarios.
6+
7+
## Test Coverage Added
8+
9+
### 1. **Edge Cases and Error Handling** (`MCP tool edge cases and error handling`)
10+
11+
- **Disconnected servers**: Verifies graceful handling when MCP server is disconnected
12+
- **Empty connections**: Tests behavior when no MCP connections are configured
13+
- **Empty tools array**: Ensures no errors when a server has no tools
14+
- **Explicit false**: Confirms tools are excluded when `allowHeadless: false` is explicit
15+
16+
### 2. **Permission Policy Interactions** (`MCP tool permission policy interactions`)
17+
18+
- **Wildcard exclude over allowHeadless**: Ensures wildcard excludes take precedence
19+
- **Specific allow vs wildcard ask**: Tests policy priority resolution
20+
- **allowHeadless with wildcard ask**: Verifies upgrade behavior in headless mode
21+
- **Wildcard ask without allowHeadless**: Confirms denial in headless mode
22+
- **argumentMatches with allowHeadless**: Tests argument-based permission matching
23+
24+
### 3. **Integration Tests** (`MCP tool integration tests`)
25+
26+
- **Mixed built-in and MCP tools**: Verifies correct filtering in headless mode
27+
- **Tool order preservation**: Ensures filtering maintains tool ordering
28+
- **Interactive to headless transition**: Tests mode switching behavior
29+
30+
### 4. **Built-in Tool Behavior**
31+
32+
- **Built-in tools in headless**: Confirms built-in tools work regardless of `allowHeadless`
33+
34+
## Test Scenarios
35+
36+
### Security-Critical Tests
37+
38+
1. ✅ Explicit exclusions always respected (cannot be bypassed by `allowHeadless`)
39+
2. ✅ Wildcard excludes take precedence over `allowHeadless`
40+
3. ✅ Default behavior is secure (tools excluded without explicit `allowHeadless: true`)
41+
42+
### Functionality Tests
43+
44+
1. ✅ Tools with `allowHeadless: true` work in headless mode
45+
2. ✅ Tools with `allowHeadless: false` are excluded in headless mode
46+
3. ✅ Tools with `allowHeadless: undefined` are excluded in headless mode
47+
4. ✅ All MCP tools work in interactive mode regardless of `allowHeadless`
48+
49+
### Integration Tests
50+
51+
1. ✅ Multiple servers with different `allowHeadless` settings
52+
2. ✅ Built-in tools always available in headless mode
53+
3. ✅ Tool enumeration maintains consistent ordering
54+
4. ✅ Mode transitions handled correctly
55+
56+
## Key Improvements Over Original Tests
57+
58+
1. **Comprehensive edge case coverage**: Tests error conditions and boundary cases
59+
2. **Permission policy interactions**: Validates complex policy scenarios with wildcards
60+
3. **Integration testing**: Ensures components work together correctly
61+
4. **Real-world scenarios**: Tests common use cases like mixed servers
62+
5. **Security validation**: Explicitly tests that security cannot be bypassed
63+
64+
## Test Organization
65+
66+
Tests are organized into 4 logical groups:
67+
68+
- `MCP tools in headless mode` (original tests from PR #9328)
69+
- `MCP tool execution permission in headless mode` (original + new built-in test)
70+
- `MCP tool edge cases and error handling` (new)
71+
- `MCP tool permission policy interactions` (new)
72+
- `MCP tool integration tests` (new)
73+
74+
## Files Modified
75+
76+
- `extensions/cli/src/stream/mcp-headless.test.ts`: Added 12 new test cases
77+
78+
## Test Execution
79+
80+
Run tests with:
81+
82+
```bash
83+
cd extensions/cli
84+
npm test -- mcp-headless.test.ts
85+
```
86+
87+
## Related PRs
88+
89+
- PR #9328: Original feature implementation and documentation
90+
- PR #9327: Parent PR for allowHeadless feature

0 commit comments

Comments
 (0)