Commit 0e4a144
authored
add anthropic caching to system prompt in agent (#1655)
# Add Anthropic Prompt Caching for Agent System Prompts
## Summary
Added Anthropic's ephemeral cache control to system prompts in
`v3AgentHandler.ts`. Refactored into a single `prependSystemMessage()`
function used by both `execute()` and `stream()`.
## Why
Reduces token costs and latency for Claude models by caching the system
prompt server-side. The `providerOptions` are ignored by non-Anthropic
models, so this is safe for all providers.
## Limitations
Currently limited to the system prompt only. Extending to conversation
messages requires restructuring message processing to insert cache
breakpoints at stable boundaries.
## Test Plan
- [x] Verify agent works with Claude and non Anthropic models
- [x] Confirm `cached_input_tokens` appears in usage metrics with Claude
<!-- This is an auto-generated description by cubic. -->
---
## Summary by cubic
Added Anthropic ephemeral caching to the agent’s system prompt to lower
Claude token usage and reduce latency. Safe for non-Anthropic models;
the options are ignored.
- **New Features**
- System messages now include Anthropic providerOptions with
cacheControl: ephemeral.
- **Refactors**
- Introduced prependSystemMessage() and used it in both text generation
and streaming to centralize system prompt handling.
<sup>Written for commit 6e2eef6.
Summary will update on new commits. <a
href="https://cubic.dev/pr/browserbase/stagehand/pull/1655">Review in
cubic</a></sup>
<!-- End of auto-generated description by cubic. -->1 parent b27c04d commit 0e4a144
1 file changed
+24
-4
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
42 | 42 | | |
43 | 43 | | |
44 | 44 | | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
45 | 67 | | |
46 | 68 | | |
47 | 69 | | |
| |||
289 | 311 | | |
290 | 312 | | |
291 | 313 | | |
292 | | - | |
293 | | - | |
| 314 | + | |
294 | 315 | | |
295 | 316 | | |
296 | 317 | | |
| |||
414 | 435 | | |
415 | 436 | | |
416 | 437 | | |
417 | | - | |
418 | | - | |
| 438 | + | |
419 | 439 | | |
420 | 440 | | |
421 | 441 | | |
| |||
0 commit comments