optimize `import niquests` performance

[tobixen]

Ref python-caldav/caldav#621 the niquests module is very much slower to load than requests. At my laptop it takes around 3x the time of requests. I had Claude investigate and it came up with some suggestions - I'll add it here shortly. Please tell if you want a pull request.

[tobixen]

Import Performance Analysis

import niquests loads 490 modules vs ~308 for requests, taking roughly 2-3x longer. Here's a breakdown of the main bottlenecks and some suggestions for improvement.

Profiling results (on a fast-ish laptop)

Component	Time	Modules
urllib3	~140ms	27
urllib3_future (on top of urllib3)	~134ms	75
qh3	~48ms	23
wassima.generate_ca_bundle()	~44ms	—
wsproto	~32ms	8
python_socks, h11, asyncio, etc.	~30ms+	66
niquests total	~410-570ms	490
requests total (for comparison)	~200-230ms	308

Bottleneck 1: wassima.generate_ca_bundle() at import time (~44ms) — unused

_constant.py:17 calls wassima.generate_ca_bundle() at module load, concatenating all system CA certificates into a 217KB string. It's marked #: Kept for BC, but it is never imported or referenced anywhere else in the codebase. This is pure waste on every import, and would be especially painful on slow machines or machines with large certificate stores.

Bottleneck 2: qh3 eagerly imported via OCSP module (~48ms)

The import chain is:

__init__.py
  → api.py (line 38: from .extensions.revocation._ocsp import InMemoryRevocationStatus)
    → extensions/revocation/_ocsp/__init__.py (line 14: from qh3._hazmat import ...)

The OCSP module unconditionally imports qh3._hazmat at the top level, which pulls in the entire qh3 package (23 modules). This isn't needed until an actual HTTPS request triggers OCSP checking.

Bottleneck 3: Dual urllib3 stack (~280ms combined)

_compat.py imports both urllib3 (~140ms) and urllib3_future (~134ms). The standard urllib3 is only checked for its version, but since it's always the "legacy" one, urllib3_future is loaded too. This is probably the hardest one to address but accounts for roughly half the total import time.

Bottleneck 4: Transitive imports from urllib3_future

urllib3_future eagerly pulls in wsproto (~32ms, 8 modules), asyncio (30 modules), python_socks (25 modules), h11 (11 modules), etc. — even if the user never uses WebSocket, async, or SOCKS features.

Suggested fixes (ordered by effort)

Fix	Est. Savings	Effort
Remove or lazy-load DEFAULT_CA_BUNDLE in _constant.py	~44ms	Trivial — it's unused
Lazy-import qh3._hazmat in OCSP module (or lazy-import the OCSP module itself in api.py)	~48ms	Low — already wrapped in try/except ImportError
Avoid importing urllib3 when urllib3_future is available (restructure _compat.py)	~140ms	Medium
Defer wsproto / WebSocket / async extensions in urllib3_future	~30ms+	Upstream change

On the machine referenced in python-caldav/caldav#621 (~10s import), these bottlenecks would be amplified proportionally — especially wassima.generate_ca_bundle() which does I/O against the system certificate store.

---

🤖 Generated with Claude Code

[tobixen]

I failed both to do a git pull before doing the analysis and to read the last comment in the original bug report. The original reporter says that his issue was resolved by upgrading to the latest wassima. Perhaps this issue should also be closed - but I leave it to @Ousret to decide :-)

[Ousret]

There some remarks that are perfectly valid.
To summarize:

• wsproto should not be loaded ideally when installed
• qh3 should only be loaded on first "need" either protocol or OCSP/CRL related usage

The rest I consider not eligible for said issue.

A quick summary of why:

• wassima.generate_ca_bundle() is used at import time but cached for the rest of the runtime. And almost certainly used.
• both urllib3 are loaded: not necessarily true and often not the case. you must opt out from the inplace fork for that to happen. so almost no one. it's going to be hard to optimize that without heuristic algorithms, and given how sensible the subject is, I'd rather not touch that until the next major where we attempt to remove this altogether (inplace fork).

and as for the title: "import niquests is slower than import requests" is perfectly normal given the canyon of features that separate us, The margin you have is what I found a year ago, and is fairly acceptable today.

---

The two items here deemed problematic will be "fixed" as soon as possible. We shouldn't expect more than 10 to 20% faster import with those, but will be welcomed anyway.

Regards,