Rename metadata field to litellm_extra_body and add custom config support

[li-boxuan]

Summary

This PR renames the metadata field to litellm_extra_body and adds configuration support for custom key-value pairs to enable direct integration with litellm's extra_body parameter.

Changes Made

🔧 Core Changes

• Renamed field: LLM.metadata → LLM.litellm_extra_body in openhands/sdk/llm/llm.py
• Updated field description to clarify its purpose for litellm's extra_body parameter
• Direct mapping: Field now maps directly to litellm's extra_body without unnecessary wrapping

📝 Usage Sites Updated

• Agent (openhands/sdk/agent/agent.py): Changed from extra_body={"metadata": self.llm.metadata} to extra_body=self.llm.litellm_extra_body
• Condenser (openhands/sdk/context/condenser/llm_summarizing_condenser.py): Same direct mapping change

⚙️ Options Handlers Enhanced

• Chat Options (openhands/sdk/llm/options/chat_options.py): Added logic to pass through litellm_extra_body when provided
• Responses Options (openhands/sdk/llm/options/responses_options.py): Added same logic for responses API

🧪 Tests Updated

• Renamed test file: test_llm_metadata.py → test_llm_litellm_extra_body.py
• Added comprehensive test coverage including custom inference cluster use cases
• All existing tests pass with the new field name

Benefits

1. Direct litellm integration: No more wrapping metadata in an extra key
2. Custom inference cluster support: Enable logging, tracking, and routing metadata
3. Flexible configuration: Support any custom key-value pairs as dict[str, Any]
4. Backward compatibility: Same data structure, just renamed field
5. Cleaner API: Direct mapping to litellm's expected parameter

Usage Example

# Before (wrapped in metadata key)
llm = LLM(model="gpt-4o", usage_id="test", metadata={"session_id": "123"})
# Internally passed as: extra_body={"metadata": {"session_id": "123"}}

# After (direct mapping)
llm = LLM(model="gpt-4o", usage_id="test", litellm_extra_body={"session_id": "123"})
# Internally passed as: extra_body={"session_id": "123"}

Custom Inference Cluster Use Case

This change enables users with custom inference clusters to pass metadata for:

• Request routing based on priority/tier
• Enhanced logging and tracing
• Custom authentication headers
• Experiment tracking
• User-specific processing

llm = LLM(
    model="gpt-4o",
    usage_id="test",
    litellm_extra_body={
        "cluster_id": "prod-cluster-1",
        "routing_key": "high-priority",
        "user_tier": "premium",
        "custom_headers": {"X-Request-Source": "openhands-agent"}
    }
)

Testing

• ✅ All existing tests pass
• ✅ New comprehensive test suite for litellm_extra_body
• ✅ Pre-commit hooks pass (formatting, linting, type checking)
• ✅ Integration tests verify proper field usage in agent and condenser

Breaking Changes

⚠️ Breaking Change: The field name has changed from metadata to litellm_extra_body. Users will need to update their code to use the new field name.

However, the functionality remains the same - it's still a dict[str, Any] that accepts custom key-value pairs.

@li-boxuan can click here to continue refining the PR

---

Agent Server images for this PR

• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server

Variants & Base Images

Variant	Base Image	Docs / Tags
golang	golang:1.21-bookworm	Link
java	eclipse-temurin:17-jdk	Link
python	nikolaik/python-nodejs:python3.12-nodejs22	Link

Pull (multi-arch manifest)

docker pull ghcr.io/openhands/agent-server:139e73b-python

Run

docker run -it --rm \
  -p 8000:8000 \
  --name agent-server-139e73b-python \
  ghcr.io/openhands/agent-server:139e73b-python

All tags pushed for this build

ghcr.io/openhands/agent-server:139e73b-golang
ghcr.io/openhands/agent-server:v1.0.0a3_golang_tag_1.21-bookworm_binary
ghcr.io/openhands/agent-server:139e73b-java
ghcr.io/openhands/agent-server:v1.0.0a3_eclipse-temurin_tag_17-jdk_binary
ghcr.io/openhands/agent-server:139e73b-python
ghcr.io/openhands/agent-server:v1.0.0a3_nikolaik_s_python-nodejs_tag_python3.12-nodejs22_binary

The 139e73b tag is a multi-arch manifest (amd64/arm64); your client pulls the right arch automatically.

[github-actions]

Coverage Report •

File	Stmts	Miss	Cover	Missing
openhands-sdk/openhands/sdk/agent
agent.py	164	58	64%	58–59, 64–65, 76, 78, 142, 146–147, 154–155, 157–159, 161–163, 179, 194–196, 203–205, 208, 213, 216–217, 220, 227, 253, 258, 288, 292, 297, 308–309, 331–333, 335, 347–348, 353–354, 366–369, 377–378, 383, 395–396, 401–402, 432, 453
openhands-sdk/openhands/sdk/context/condenser
llm_summarizing_condenser.py	45	25	44%	22, 29, 32–34, 37–38, 41, 43, 45–49, 52, 55, 57, 64, 66, 71–75, 77
openhands-sdk/openhands/sdk/llm
llm.py	441	170	61%	276, 286, 294, 298, 303, 307, 319, 323–325, 329–330, 341, 343, 347, 364, 388, 398, 403, 407, 412, 423, 439, 460, 464, 479, 485–486, 505–506, 514, 539–541, 562–563, 566, 570, 582, 587–590, 597, 600, 608–613, 616, 631, 635–637, 639–640, 645–646, 648, 655, 658–660, 722–725, 730–735, 739–740, 749–751, 754–755, 792–793, 839, 859, 894, 910, 913–915, 918–926, 930–932, 935, 938–940, 947–948, 957, 964–966, 970, 972–977, 979–996, 999–1003, 1005–1006, 1012–1021, 1034, 1048, 1053, 1067–1068, 1073–1074, 1078, 1080, 1089–1090, 1094, 1099–1100, 1105
openhands-sdk/openhands/sdk/llm/options
chat_options.py	37	21	43%	28–29, 34–35, 37–38, 40–42, 46–47, 52, 54, 56–57, 61–63, 67–68, 72
responses_options.py	21	17	19%	17, 25–26, 29–30, 32, 35–40, 43–44, 47–48, 50
TOTAL	7991	3428	57%

[enyst]

Hah, Sonnet! 😅

This test may be useful if it tested what we actually send to litellm completion() here I guess

[enyst]

Just to note, I'm pretty sure this is correct. It's not really for LLM itself, it's a field for additional logging / tracking, specially when these folks trace LLMs on the cloud

[blacksmith-sh]

[Automatic Post]: It has been a while since there was any activity on this PR. @li-boxuan, are you still working on it? If so, please go ahead, if not then please request review, close it, or request that someone else follow up.

[blacksmith-sh]

[Automatic Post]: I have assigned @enyst as a reviewer based on git blame information. Thanks in advance for the help!