Skip to content

telemetry: make_http_sink docstring overstates backpressure — max_clients caps concurrency, not queue depth #946

Description

@paddymul

Problem

buckaroo/server/telemetry.py:13-14 claims AsyncHTTPClient's max_clients "caps in-flight requests, so a dead companion can't grow an unbounded backlog." max_clients bounds only how many fetches run concurrently — it does not bound how many are queued. When the sink's _sink calls loop.spawn_callback(_post, record) faster than the 2s-timeout POSTs drain (companion down or hung), the excess fetches pile up in SimpleAsyncHTTPClient's internal pending deque, which is unbounded; each queued fetch also retains its record dict. So a dead companion under sustained span emission grows memory without bound, contradicting the docstring's stated guarantee.

In practice the emission volume is low — only the firstpull.* spans per load, not per-scroll — so the queue drains between loads and the growth is latent rather than active today. The defect is the false guarantee in the docstring plus the missing real bound, not an observed leak.

Suggested fix

Either (a) make the bound real — track an in-flight count in the closure and drop records past a cap (telemetry is best-effort, so dropping is acceptable), or (b) soften the docstring to state the queue is unbounded and that today's safety rests on low emission volume + the 2s timeout, with a note to add a cap if per-row spans land (the deferred v2).

Context

Identified during review of #944 (the server-side cache/stats telemetry signal). The sink and its docstring were added in #943/#944. No observed incident; flagged as a latent bound the docstring currently misrepresents.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions