Skip to content

fix: do not inject agent tools into title-generation requests - #102

Closed
zhanglei wants to merge 18 commits into
jasonxu114514:mainfrom
zhanglei:fix/title-generation-no-tools
Closed

zhanglei wants to merge 18 commits into
jasonxu114514:mainfrom
zhanglei:fix/title-generation-no-tools

Conversation

@zhanglei

Copy link
Copy Markdown

OpenCode's own session-title call is sent with an empty tool list (packages/opencode/src/session/prompt.ts builds it as { small: true, tools: {}, messages: [{ role: "user", content: "Generate a title for this conversation:\n" }, ...] }). The free-tier gate accepts that tool-free form, but rejects the same call with 403 FreeTierError once an agent tool list is attached.

The gateway treated every free-model body as agent traffic and injected the core tool set, so a title-generation request that works inside OpenCode failed with "OpenCode's free tier can only be used from within OpenCode" as soon as it was routed through the gateway.

Detect the title prompt before shaping and leave such a request tool-free. The match is narrow — a user turn whose text begins with the verbatim title prompt — because a false positive would strip tools from real agent traffic and produce the same 403.

OpenCode's own session-title call is sent with an empty tool list
(packages/opencode/src/session/prompt.ts builds it as
{ small: true, tools: {}, messages: [{ role: "user", content: "Generate a
title for this conversation:\n" }, ...] }). The free-tier gate accepts that
tool-free form, but rejects the same call with 403 FreeTierError once an agent
tool list is attached.

The gateway treated every free-model body as agent traffic and injected the
core tool set, so a title-generation request that works inside OpenCode failed
with "OpenCode's free tier can only be used from within OpenCode" as soon as it
was routed through the gateway.

Detect the title prompt before shaping and leave such a request tool-free.
The match is narrow — a user turn whose text begins with the verbatim title
prompt — because a false positive would strip tools from real agent traffic
and produce the same 403.
zhanglei and others added 17 commits October 17, 2026 09:42
Resolve the conflict in prepareAnonymousBody: main added the legacy /zen/v1
branch, the PR added title-generation detection. Combine both so title
generation stays tool-free on both gates.
…no-tools

# Conflicts:
#	internal/gateway/upstream.go
The upstream now tries the v2 /inference lane before v1, so the note
justifying the guard by lane order no longer describes the code. The
guard itself still applies to both gates.
…no-tools

# Conflicts:
#	internal/gateway/upstream.go
Registrations were paced by a fixed 12-second gap, which made a cold pool
and a 429 replacement wait minutes for capacity. Remove the gap: the
registrar now issues registrations back to back in the background,
serialized by the pool's create mutex, and only a rejected attempt backs
off (doJSON still honours Retry-After).

Keep half of warp.pool_size (at least one) hot as a preload: the first
exit still gates readiness, the rest register in the background, and
desiredLocked/scaleDown treat the preload as a floor so idle surplus is
still retired after 60 seconds but the warm bench survives. A 429 or 403
removes the exit from the ring and the next request uses a preloaded exit
immediately.

/healthz reports warp.preload, and the WebUI and all four READMEs
describe the behaviour.
A Cloudflare 429 on /reg no longer sleeps inside the registration path:
the registrar records the Retry-After window and fails fast, and the pool
schedules its next attempt after that window. Registration can no longer
hold createMu for minutes while requests wait for an exit.

Refill backs off when the colo hands out fewer distinct egress blocks
than the preload: three consecutive duplicate registrations pause it for
1-5 minutes instead of registering into a saturated colo forever.

Automatic free-model probes skip a pool with fewer than two exits and
stop the sweep after one probe retires an exit, so they cannot drain the
warm bench. Diagnostics are unaffected.
Startup registers the first exit synchronously and failed closed on any
error, including Cloudflare's rate limit — which a fresh process can still
receive because the limit follows the source address. Wait out the pause
inside the initialization deadline instead, and give WARP initialization
one full rate-limit window so a deploy does not fail on a limit that is
about to expire.
…no-tools

Resolve the internal/gateway/upstream.go conflict by keeping origin/main's
rewritten tool shaping (clint_ prefix plus the complete official tool set,
returning *wire.ToolRenames) and retaining this branch's title-generation
gate: a title call still skips renameAndCompleteTools so the free tier sees
the tool-free body OpenCode itself sends.
The anonymous free tier now calls the configured v2 /inference origin only; the retired v1 /zen fallback is commented out rather than deleted, so the lane order can be restored in one edit. Catalog refresh and probes follow anonymousUpstreams, so they stop trying /zen too.

A request whose client names no project now gets a fresh 40-hex project id. The free tier ledges quota per project id and egress, and the old per-caller default was one shared bucket that popular exits had already burned.

Response tool names now resolve in the order exact wire name, clint_ prefix, case fold, separator fold. A fold two declared tools share is never guessed between. The stream and JSON rewriters share the resolver, and their fast path now recognizes unprefixed tool-call frames.
after() added to the pool wait group outside the lock, so a timer scheduled just as Close began Wait could be missed by the shutdown wait. Increment under the lock and refuse to schedule after Close, which removes the Add/Wait race.
…no-tools

# Conflicts:
#	internal/gateway/upstream.go
#	internal/protocol/tools.go
@jasonxu114514

Copy link
Copy Markdown
Owner

Superseded by #169, which is now on main and also stops tools being added to title-generation requests. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants