Skip to content

Dynamic attributes - #605

Open
gyozaaaa wants to merge 6 commits into
0x676e67:mainfrom
gyozaaaa:dynamic-attributes
Open

Dynamic attributes#605
gyozaaaa wants to merge 6 commits into
0x676e67:mainfrom
gyozaaaa:dynamic-attributes

Conversation

@gyozaaaa

Copy link
Copy Markdown

Dynamic headers and proxies on an instantiated client

Both settings could only be given to the constructor and were fixed for the life of the
client. They are now readable and writable attributes on Client and
blocking.Client.

client = Client(headers={"x-api-key": "secret"}, proxies=Proxy.all("http://a:8080"))

client.proxies = Proxy.all("http://b:8080")     # a single proxy or a sequence
client.proxies = None                            # back to the system proxies

client.headers.update({"x-request-id": "1"})     # add, or replace what is there
client.headers["x-trace"] = "abc"
del client.headers["x-trace"]
client.headers = {"x-api-key": "other"}          # replace them altogether
  • proxies reads back as a list, or None. Proxy.all(...) is accepted wherever a
    sequence is, on the attribute and on the constructor argument alike.
  • headers reads back a ClientHeaders, a HeaderMap bound to the client, so the
    usual lookups work on it and it can be handed to another client. HeaderMap.update()
    is new and available on plain maps too.
  • Changing either keeps the rest of the configuration and the cookie jar. Only the
    headers of the client are replaced, so those coming from an emulation are kept.
    Requests already in flight keep the settings they started with.

Implementation

wreq::Client bakes default headers and proxies in at build time, so a change rebuilds
the underlying client from the retained configuration and swaps it in through an
ArcSwap. The request path stays lock-free; a mutex serializes the rebuilds so that
concurrent updates of the headers and the proxies cannot lose each other. Dropping the
old connection pool is intended: connections opened through the previous proxy are not
reused.

Two details worth flagging for review:

  • Builder.user_agent moved from PyBackedStr to String, because PyBackedStr is not
    Clone in pyo3 0.29 and the retained configuration has to be cloneable. One small
    allocation per client build.
  • A tls_verify certificate file is now read once, at construction, rather than on every
    rebuild.

Tests

tests/proxy_test.py and the client half of tests/header_test.py are hermetic — local
servers standing in for the origin and for two forward proxies, no network. They cover
switching, clearing, scheme-specific proxy lists, per-request override, configuration and
cookie-jar preservation, emulation headers surviving a header change, concurrent updates
from threads, and the blocking client.

gyozaaaa and others added 6 commits August 30, 2026 14:02
a list for both sync and async clients. Note that reading the
`proxies` value list will always be a list, similar to how `Cookies`
currently works.
A `Client` keeps its configuration around to rebuild itself whenever its headers
or proxies change, and `tls_verify` was left in it as a `CertificatePath`. Every
rebuild therefore re-read the file from the disk, so a plain
`client.headers["x"] = "1"` failed with an I/O error once the file was gone, and
silently switched trust roots when the file had been swapped in the meantime.

Resolve the path into a `CertStore` in `Client::new`, the way an in-memory store
given to `tls_verify` is already reused across rebuilds.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`Client.headers` handed out a snapshot of the headers taken outside of the
rebuild lock, so `client.headers.update(...)` was a read-modify-write whose read
was unlocked. Two threads could snapshot the same map, mutate their own copies
and serialize only on the store, which replaces the header set wholesale: the
update of whichever thread stored first was dropped without a trace. The lock
only ever covered the `headers` and `proxies` setters, not the view the guide
teaches as the way to change the headers of a client.

Express each mutation as a `HeaderUpdate` and replay it, under the lock, onto
the headers read back from the client, then refresh the view with the result. A
view that has gone stale now updates the current headers instead of restoring
the ones it was created with. `HeaderMap` applies the same `HeaderUpdate`, so
both share one implementation of every operation.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The test only inspects the headers of a client, it never makes a request, so
the fixture started and tore down an HTTP server for nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant