Skip to content

Commit 8393004

Browse files
ashwin-antclaudekm-anthropic
authored
Add custom tool callbacks and e2e tests (#157)
## Summary This PR adds support for custom tool callbacks and comprehensive e2e testing for MCP calculator functionality. ## Key Features Added - **Custom tool permission callbacks** - Allow dynamic tool permission control via `can_use_tool` callback - **E2E test suite** - Real Claude API tests validating MCP tool execution end-to-end - **Fixed MCP calculator example** - Now properly uses `allowed_tools` for permission management ## Changes ### Custom Callbacks - Added `ToolPermissionContext` and `PermissionResult` types for tool permission handling - Implemented `can_use_tool` callback support in SDK client - Added comprehensive tests in `tests/test_tool_callbacks.py` ### E2E Testing Infrastructure - Created `e2e-tests/` directory with pytest-based test suite - `test_mcp_calculator.py` - Tests all calculator operations with real API calls - `conftest.py` - Pytest config with mandatory API key validation - GitHub Actions workflow for automated e2e testing on main branch - Comprehensive documentation in `e2e-tests/README.md` ### Bug Fixes - Fixed MCP calculator example to use `allowed_tools` instead of incorrect `permission_mode` - Resolved tool permission issues preventing MCP tools from executing ## Testing E2E tests require `ANTHROPIC_API_KEY` environment variable and will fail without it. Run locally: ```bash export ANTHROPIC_API_KEY=your-key python -m pytest e2e-tests/ -v -m e2e ``` Run unit tests including callback tests: ```bash python -m pytest tests/test_tool_callbacks.py -v ``` 🤖 Generated with [Claude Code](https://claude.ai/code) --------- Co-authored-by: Claude <noreply@anthropic.com> Co-authored-by: Kashyap Murali <kashyap@anthropic.com>
1 parent d3190f1 commit 8393004

15 files changed

Lines changed: 1302 additions & 91 deletions

File tree

‎.github/workflows/test-e2e.yml‎

Lines changed: 0 additions & 45 deletions
This file was deleted.

‎.github/workflows/test.yml‎

Lines changed: 93 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -4,34 +4,103 @@ on:
44
pull_request:
55
push:
66
branches:
7-
- 'main'
7+
- "main"
88

99
jobs:
1010
test:
1111
runs-on: ubuntu-latest
1212
strategy:
1313
matrix:
14-
python-version: ['3.10', '3.11', '3.12', '3.13']
15-
14+
python-version: ["3.10", "3.11", "3.12", "3.13"]
15+
16+
steps:
17+
- uses: actions/checkout@v4
18+
19+
- name: Set up Python ${{ matrix.python-version }}
20+
uses: actions/setup-python@v5
21+
with:
22+
python-version: ${{ matrix.python-version }}
23+
24+
- name: Install dependencies
25+
run: |
26+
python -m pip install --upgrade pip
27+
pip install -e ".[dev]"
28+
29+
- name: Run tests
30+
run: |
31+
python -m pytest tests/ -v --cov=claude_code_sdk --cov-report=xml
32+
33+
- name: Upload coverage to Codecov
34+
uses: codecov/codecov-action@v4
35+
with:
36+
file: ./coverage.xml
37+
fail_ci_if_error: false
38+
39+
test-e2e:
40+
runs-on: ubuntu-latest
41+
needs: test # Run after unit tests pass
42+
strategy:
43+
matrix:
44+
python-version: ["3.10", "3.11", "3.12", "3.13"]
45+
1646
steps:
17-
- uses: actions/checkout@v4
18-
19-
- name: Set up Python ${{ matrix.python-version }}
20-
uses: actions/setup-python@v5
21-
with:
22-
python-version: ${{ matrix.python-version }}
23-
24-
- name: Install dependencies
25-
run: |
26-
python -m pip install --upgrade pip
27-
pip install -e ".[dev]"
28-
29-
- name: Run tests
30-
run: |
31-
python -m pytest tests/ -v --cov=claude_code_sdk --cov-report=xml
32-
33-
- name: Upload coverage to Codecov
34-
uses: codecov/codecov-action@v4
35-
with:
36-
file: ./coverage.xml
37-
fail_ci_if_error: false
47+
- uses: actions/checkout@v4
48+
49+
- name: Set up Python ${{ matrix.python-version }}
50+
uses: actions/setup-python@v5
51+
with:
52+
python-version: ${{ matrix.python-version }}
53+
54+
- name: Install Claude Code
55+
run: |
56+
curl -fsSL https://claude.ai/install.sh | bash
57+
echo "$HOME/.local/bin" >> $GITHUB_PATH
58+
59+
- name: Verify Claude Code installation
60+
run: claude -v
61+
62+
- name: Install dependencies
63+
run: |
64+
python -m pip install --upgrade pip
65+
pip install -e ".[dev]"
66+
67+
- name: Run end-to-end tests with real API
68+
env:
69+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
70+
run: |
71+
python -m pytest e2e-tests/ -v -m e2e
72+
73+
test-examples:
74+
runs-on: ubuntu-latest
75+
needs: test-e2e # Run after e2e tests
76+
strategy:
77+
matrix:
78+
python-version: ["3.10", "3.11", "3.12", "3.13"]
79+
80+
steps:
81+
- uses: actions/checkout@v4
82+
83+
- name: Set up Python ${{ matrix.python-version }}
84+
uses: actions/setup-python@v5
85+
with:
86+
python-version: ${{ matrix.python-version }}
87+
88+
- name: Install Claude Code
89+
run: |
90+
curl -fsSL https://claude.ai/install.sh | bash
91+
echo "$HOME/.local/bin" >> $GITHUB_PATH
92+
93+
- name: Verify Claude Code installation
94+
run: claude -v
95+
96+
- name: Install dependencies
97+
run: |
98+
python -m pip install --upgrade pip
99+
pip install -e .
100+
101+
- name: Run example scripts
102+
env:
103+
ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
104+
run: |
105+
python examples/quick_start.py
106+
timeout 120 python examples/streaming_mode.py all

‎README.md‎

Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,90 @@ options = ClaudeCodeOptions(
7676
)
7777
```
7878

79+
### SDK MCP Servers (In-Process)
80+
81+
The SDK now supports in-process MCP servers that run directly within your Python application, eliminating the need for separate processes.
82+
83+
#### Creating a Simple Tool
84+
85+
```python
86+
from claude_code_sdk import tool, create_sdk_mcp_server
87+
88+
# Define a tool using the @tool decorator
89+
@tool("greet", "Greet a user", {"name": str})
90+
async def greet_user(args):
91+
return {
92+
"content": [
93+
{"type": "text", "text": f"Hello, {args['name']}!"}
94+
]
95+
}
96+
97+
# Create an SDK MCP server
98+
server = create_sdk_mcp_server(
99+
name="my-tools",
100+
version="1.0.0",
101+
tools=[greet_user]
102+
)
103+
104+
# Use it with Claude
105+
options = ClaudeCodeOptions(
106+
mcp_servers={"tools": server}
107+
)
108+
109+
async for message in query(prompt="Greet Alice", options=options):
110+
print(message)
111+
```
112+
113+
#### Benefits Over External MCP Servers
114+
115+
- **No subprocess management** - Runs in the same process as your application
116+
- **Better performance** - No IPC overhead for tool calls
117+
- **Simpler deployment** - Single Python process instead of multiple
118+
- **Easier debugging** - All code runs in the same process
119+
- **Type safety** - Direct Python function calls with type hints
120+
121+
#### Migration from External Servers
122+
123+
```python
124+
# BEFORE: External MCP server (separate process)
125+
options = ClaudeCodeOptions(
126+
mcp_servers={
127+
"calculator": {
128+
"type": "stdio",
129+
"command": "python",
130+
"args": ["-m", "calculator_server"]
131+
}
132+
}
133+
)
134+
135+
# AFTER: SDK MCP server (in-process)
136+
from my_tools import add, subtract # Your tool functions
137+
138+
calculator = create_sdk_mcp_server(
139+
name="calculator",
140+
tools=[add, subtract]
141+
)
142+
143+
options = ClaudeCodeOptions(
144+
mcp_servers={"calculator": calculator}
145+
)
146+
```
147+
148+
#### Mixed Server Support
149+
150+
You can use both SDK and external MCP servers together:
151+
152+
```python
153+
options = ClaudeCodeOptions(
154+
mcp_servers={
155+
"internal": sdk_server, # In-process SDK server
156+
"external": { # External subprocess server
157+
"type": "stdio",
158+
"command": "external-server"
159+
}
160+
}
161+
)
162+
```
79163

80164
## API Reference
81165

‎e2e-tests/README.md‎

Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
# End-to-End Tests for Claude Code SDK
2+
3+
This directory contains end-to-end tests that run against the actual Claude API to verify real-world functionality.
4+
5+
## Requirements
6+
7+
### API Key (REQUIRED)
8+
9+
These tests require a valid Anthropic API key. The tests will **fail** if `ANTHROPIC_API_KEY` is not set.
10+
11+
Set your API key before running tests:
12+
13+
```bash
14+
export ANTHROPIC_API_KEY="your-api-key-here"
15+
```
16+
17+
### Dependencies
18+
19+
Install the development dependencies:
20+
21+
```bash
22+
pip install -e ".[dev]"
23+
```
24+
25+
## Running the Tests
26+
27+
### Run all e2e tests:
28+
29+
```bash
30+
python -m pytest e2e-tests/ -v
31+
```
32+
33+
### Run with e2e marker only:
34+
35+
```bash
36+
python -m pytest e2e-tests/ -v -m e2e
37+
```
38+
39+
### Run a specific test:
40+
41+
```bash
42+
python -m pytest e2e-tests/test_mcp_calculator.py::test_basic_addition -v
43+
```
44+
45+
## Cost Considerations
46+
47+
⚠️ **Important**: These tests make actual API calls to Claude, which incur costs based on your Anthropic pricing plan.
48+
49+
- Each test typically uses 1-3 API calls
50+
- Tests use simple prompts to minimize token usage
51+
- The complete test suite should cost less than $0.10 to run
52+
53+
## Test Coverage
54+
55+
### MCP Calculator Tests (`test_mcp_calculator.py`)
56+
57+
Tests the MCP (Model Context Protocol) integration with calculator tools:
58+
59+
- **test_basic_addition**: Verifies the add tool executes correctly
60+
- **test_division**: Tests division with decimal results
61+
- **test_square_root**: Validates square root calculations
62+
- **test_power**: Tests exponentiation
63+
- **test_multi_step_calculation**: Verifies multiple tools can be used in sequence
64+
- **test_tool_permissions_enforced**: Ensures permission system works correctly
65+
66+
Each test validates:
67+
1. Tools are actually called (ToolUseBlock present in response)
68+
2. Correct tool inputs are provided
69+
3. Expected results are returned
70+
4. Permission system is enforced
71+
72+
## CI/CD Integration
73+
74+
These tests run automatically on:
75+
- Pushes to `main` branch (via GitHub Actions)
76+
- Manual workflow dispatch
77+
78+
The workflow uses `ANTHROPIC_API_KEY` from GitHub Secrets.
79+
80+
## Troubleshooting
81+
82+
### "ANTHROPIC_API_KEY environment variable is required" error
83+
- Set your API key: `export ANTHROPIC_API_KEY=sk-ant-...`
84+
- The tests will not skip - they require the key to run
85+
86+
### Tests timing out
87+
- Check your API key is valid and has quota available
88+
- Ensure network connectivity to api.anthropic.com
89+
90+
### Permission denied errors
91+
- Verify the `allowed_tools` parameter includes the necessary MCP tools
92+
- Check that tool names match the expected format (e.g., `mcp__calc__add`)
93+
94+
## Adding New E2E Tests
95+
96+
When adding new e2e tests:
97+
98+
1. Mark tests with `@pytest.mark.e2e` decorator
99+
2. Use the `api_key` fixture to ensure API key is available
100+
3. Keep prompts simple to minimize costs
101+
4. Verify actual tool execution, not just mocked responses
102+
5. Document any special setup requirements in this README

0 commit comments

Comments
 (0)