Skip to content

Use multithreaded compression for bigger responses 512kb/4mb - #6637

Open
ResuBaka wants to merge 2 commits into
clockworklabs:masterfrom
ResuBaka:compression/multithreading
Open

ResuBaka wants to merge 2 commits into
clockworklabs:masterfrom
ResuBaka:compression/multithreading

Conversation

@ResuBaka

Copy link
Copy Markdown
Contributor

Description of Changes

This add a multithreading to botli and zstd. Which is usefull for bigger messages to get the faster to clients and/or reduce tmp memory usage.

Snapshot size Brotli Q1: sync → parallel Zstd fast-2: sync → parallel
4 MiB 80.1 → 43.9 ms (1.83×) 6.46 → 3.68 ms (1.76×)
16 MiB 318 → 174 ms (1.83×) 27.7 → 12.9 ms (2.15×)
64 MiB 1,264 → 353 ms (3.58×) 110 → 27 ms (4.08×)

API and ABI breaking changes

n/a

Rollback safety impact

This change is save to rollback.

Expected complexity level and risk

The complexity is probably 2 as it could slighlty increase cpu usage spikes in favor of faster responses.

Testing

It was tested via building some benchmark code to check speed improvments against running the old none threaded way.

@ResuBaka

Copy link
Copy Markdown
Contributor Author

This is needs #6634 as it is build on top of it.

@ResuBaka ResuBaka changed the title Compression/multithreading Use multithreaded compression for bigger responses 512kb/4mb Nov 19, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant