This report measures bunqueue queue and Workflow Engine performance on one native high-end Linux host. It is an engineering capacity study, not a single headline contest: every row names the operation, topology, persistence mode, sample count, aggregation, and integrity boundary.
The representative results are:
| Workload | Representative result |
|---|---|
| Internal in-memory batched push, 1M jobs | 729,395 jobs/s median |
| Internal in-memory full lifecycle, 1M jobs | 311,915 jobs/s median |
Public Embedded on-disk sustained addBulk, 50K cell |
186,384 jobs/s median |
TCP PUSHB, fresh broker/database, 50K jobs |
158,779 jobs/s median |
| TCP no-work worker drain, concurrency 50 | 17,256 jobs/s median |
| Sequential durable Embedded/TCP add | 60,835 / 27,191 ops/s median |
| Single Workflow Engine, linear Embedded/TCP | 2,700 / 3,187 workflows/s median |
| 12 tuned Workflow Engines, Embedded/TCP | 25,873 / 17,496 workflows/s median |
The Workflow Engine campaign reconciled 63,000 measured single-engine executions and 750,000 measured horizontal executions. Including discarded warm-ups, the horizontal runner checked 1,000,000 workflow executions. Every sample passed terminal-state and event conservation.
| Field | Value |
|---|---|
| Date | 2026-07-30, Europe/Rome |
| Engine revision | af027f04d2064b701ee2243eac737d74b8d87706 |
| Runtime source changes during measurement | None |
| Bun | 1.3.14, x64 |
| OS | Linux 7.0.0-14 |
| CPU | AMD Ryzen 9 9950X3D, 16 physical cores / 32 threads |
| Memory | 59 GiB |
| CPU governor | performance |
| Virtualization | None detected |
The Workflow benchmark runner was added to the candidate worktree after the queue campaign; it does not modify runtime source. All performance measurements ran natively. Docker was used only for the functional sandbox, which completed before native performance runs.
An initial exploratory run was excluded before aggregation because it overlapped an existing Docker sandbox and did not isolate the Embedded data path. No number from that run appears below.
- Each primary single-instance row uses one discarded warm-up process followed by repeated measured fresh processes.
- The 1M-job queue campaign uses 21 measured runs.
- TCP serde, TCP lifecycle, and durable campaigns use 21 measured runs.
- Workflow linear uses 21 measured runs per mode; parallel, compensation, and signal use 7 per mode.
- Horizontal campaigns use 3 measured barriers after one discarded barrier.
- Median is the representative value. Tables also show p05/p95 or min/max and population coefficient of variation (CV).
- Latency percentiles are calculated inside each run. Workflow tables report the median of run-level percentiles.
Every measured Workflow sample uses:
- a new Bun process;
- a unique queue and workflow name;
- a fresh temporary directory;
- a fresh SQLite workflow database;
- in TCP mode, a separate fresh broker process, broker database, and dynamic TCP/HTTP ports;
- explicit close and removal of database, WAL, and SHM files.
Queue campaigns used the same fresh-process/state discipline in their native orchestration. The public API sustained curve intentionally grows one database through its 1K, 5K, 10K, and 50K cells; that exception is part of the workload, not hidden state.
Timing is accepted only with the matching invariants:
- queue campaigns reconcile pushed, pulled, completed, duplicate, missing, and invalid job IDs plus final counts;
- workflow campaigns inspect every persisted execution;
- linear, parallel, and signal runs must be
completed; - compensation runs must be
failedwithrollbackStatus='completed'and both reversals recorded exactly once; - lifecycle, step, waiting, signal, failure, and compensation events must conserve exactly.
This is src/benchmark/million-jobs.bench.ts: 16 queues and 16 workers over the
internal batched QueueManager API, removeOnComplete:true, no dataPath.
It measures the in-memory engine, not public Queue.addBulk and not SQLite
write throughput.
| Phase | Samples | Median | Min–max | CV |
|---|---|---|---|---|
| Push | 21 | 729,395 jobs/s | 693,001–747,943 | 1.94% |
| Process | 21 | 541,712 jobs/s | 523,286–558,036 | 1.40% |
| Complete lifecycle | 21 | 311,915 jobs/s | 303,674–316,556 | 1.21% |
All 21M jobs passed pull-time and completed-event integrity: no missing index, duplicate ID, or invalid payload.
bench:pushbulk was run in eight fresh campaigns; the first was discarded.
Each cell is already the median of three repetitions. The table is the median
of the seven campaign medians for the final 50K cell.
| Mode | add() |
addBulk() |
|---|---|---|
| Embedded, on-disk SQLite | 147,818 jobs/s | 186,384 jobs/s |
| TCP, on-disk SQLite | 127,476 jobs/s | 87,319 jobs/s |
Each campaign executes the lower scales first against the same database.
Therefore this is a sustained/grown-database result. It must not be compared
directly with a fresh-database PUSHB microbenchmark.
BENCH_N=50000 BENCH_RUNS=21 bun run bench:tcp used a fresh server and database
for each operation.
| Operation | Median | Min–max | CV |
|---|---|---|---|
| Pipelined individual push, auto-batch disabled | 80,978 ops/s | 78,331–83,806 | 2.06% |
PUSHB, 50K jobs |
158,779 jobs/s | 146,496–163,501 | 3.25% |
The final process then measured 5,000 sequential localhost round trips:
| Operation | p50 | p99 |
|---|---|---|
| Buffered TCP add | 14 µs | 292 µs |
The primary lifecycle campaign preloaded 20K jobs, then drained them with one TCP worker at concurrency 50. One warm-up and 21 fresh measured server/database pairs produced:
| Metric | Result |
|---|---|
| Median | 17,256 jobs/s |
| p05 / p95 | 16,455 / 17,864 jobs/s |
| CV | 2.86% |
| Median / p95 duration | 1,159 / 1,215 ms |
All 420K jobs completed exactly once and final queue counts were clean.
The separate 10K-job sweep used one warm-up plus seven measured fresh samples per point:
| Worker concurrency | Median drain |
|---|---|
| 32 | 17,607 jobs/s |
| 48 | 17,541 jobs/s |
| 64 | 17,191 jobs/s |
| 96 | 15,830 jobs/s |
| 128 | 15,094 jobs/s |
| 192 | 14,081 jobs/s |
The useful knee for this localhost no-work processor is 32–48. More concurrency reduces throughput.
Each of 21 fresh processes measured 2,000 durable:true sequential adds after
50 warm-up operations.
| Mode | Throughput median | p05 / p95 | CV | Run-median latency p50 / p95 / p99 |
|---|---|---|---|---|
| Embedded | 60,835 ops/s | 53,698 / 63,002 | 7.83% | 15 / 25 / 59 µs |
| TCP | 27,191 ops/s | 21,590 / 29,089 | 8.90% | 30 / 57 / 120 µs |
All 43,050 operations per mode, including warm-ups, were present in queue counts.
Twelve independent processes each executed the 1M-job lifecycle against their own state. Three barriers produced 2,022,926, 2,008,704, and 1,996,340 jobs/s.
Median aggregate throughput was 2,008,704 jobs/s. The campaign reconciled 36M/36M jobs. Peak combined RSS was about 37.4 GiB, dominated by the twelve 1M-ID integrity sets; sampled CPU peaked near 2,513% and swap stayed unused.
Twelve broker/client pairs each preloaded 50K jobs and started workers from one barrier.
| Campaign | Conservative barrier | Exact drain | Integrity |
|---|---|---|---|
| 1 | 46,529 jobs/s | 46,937 jobs/s | 600K / 600K |
| 2 | 46,324 jobs/s | 46,729 jobs/s | 600K / 600K |
| 3 | 46,645 jobs/s | 47,054 jobs/s | 600K / 600K |
Median exact drain was 46,937 jobs/s. Peak combined RSS was about 5.2 GiB and sampled CPU about 2,249%. The 2.7× gain over one pair is the observed host capacity; multiplying a single-pair peak by 12 would be wrong.
| Scenario | Execution graph | Correctness boundary |
|---|---|---|
| Linear | validate → transform → persist | 3 steps, terminal completed |
| Parallel | prepare → 3 inline Promise.all steps → join |
5 step completions, terminal completed |
| Compensation | reserve → charge → intentional failure | terminal failed, two reverse compensations |
| Signal | request → wait for approved → finish |
waiting and signal exactly once, then completed |
Parallel steps execute inside one workflow node job; this scenario measures Workflow Engine parallel bookkeeping and join behavior, not distributed child workers.
The concurrency sweep at the same 1,000-execution linear scale found an Embedded plateau at 64–128 and a TCP plateau at 32–64. Final configurations were Embedded 128 and TCP 64. TCP 64 was confirmed over 21 samples at 3,187 workflows/s (CV 1.77%), versus TCP 32 at 3,020 (CV 1.72%).
| Mode/scenario | Runs × executions | Throughput median | p05 / p95 | CV | Run-median latency p50 / p95 / p99 |
|---|---|---|---|---|---|
| Embedded linear | 21 × 1,000 | 2,700 wf/s | 2,570 / 2,792 | 2.86% | 305,564 / 342,783 / 346,139 µs |
| Embedded parallel | 7 × 500 | 2,118 wf/s | 1,997 / 2,173 | 2.70% | 202,750 / 219,812 / 221,523 µs |
| Embedded compensation | 7 × 500 | 2,055 wf/s | 2,005 / 2,118 | 2.05% | 199,776 / 225,617 / 228,057 µs |
| Embedded signal | 7 × 500 | 1,928 wf/s | 1,883 / 1,961 | 1.39% | 71,455 / 84,673 / 86,378 µs resume |
| TCP linear | 21 × 1,000 | 3,187 wf/s | 3,075 / 3,261 | 1.77% | 260,412 / 284,289 / 288,198 µs |
| TCP parallel | 7 × 500 | 2,456 wf/s | 2,368 / 2,528 | 2.48% | 172,855 / 186,354 / 186,917 µs |
| TCP compensation | 7 × 500 | 2,239 wf/s | 2,218 / 2,384 | 2.87% | 177,362 / 200,878 / 204,270 µs |
| TCP signal | 7 × 500 | 2,234 wf/s | 2,204 / 2,306 | 1.72% | 56,576 / 64,922 / 65,729 µs resume |
These are saturated-batch workflow:started-event-to-terminal latencies: an
execution includes time waiting behind other executions after its lifecycle
event begins. The throughput interval is broader and starts before the first
Engine.start() call. Neither metric is idle single-workflow service latency.
TCP being faster in this workload does not mean the network is free. The TCP topology gives the broker and Workflow Store separate processes and separate SQLite files; Embedded shares one file across the store and queue connections. That extra parallelism outweighs localhost protocol cost for these no-op steps.
For the signal phase:
| Mode | Park throughput median | Resume throughput median | Park latency p50 / p95 run medians | Resume latency p50 / p95 |
|---|---|---|---|---|
| Embedded | 3,205 wf/s | 4,837 wf/s | 127,060 / 139,070 µs | 71,455 / 84,673 µs |
| TCP | 3,518 wf/s | 6,149 wf/s | 113,533 / 122,988 µs | 56,576 / 64,922 µs |
Across both modes, the measured single-engine campaign contains:
- 63,000 workflow executions and
workflow:startedevents; - 189,000 completed step events;
- 56,000 successful workflow completions;
- 7,000 intentional workflow failures;
- 14,000 successful compensation outcomes.
The TCP Workflow Engine has an important operational boundary. The broker limits each protocol client to 10,000 requests per 60-second window by default. One workflow expands into multiple queue commands.
| Linear executions | Default-cap result |
|---|---|
| 1,000 | 3,214 wf/s; 311 ms |
| 2,000 | 3,769 wf/s; 531 ms |
| 3,000 | 4,005 wf/s; 749 ms |
| 3,500 | 58 wf/s; 60,159 ms; p95 60,088 ms |
The 3,500-execution result reproduced. During the plateau, read-only SQLite inspection showed:
- workflow store: 2,604 completed, 896 running at node index 2;
- broker: 9,570 completed jobs, 896 waiting, 34 active.
At window turnover the process emitted Rate limit exceeded for ACK batches,
then completed all executions. Lowering TCP partial-frame timeout, client
command timeout, or worker cleanup interval to 5 seconds did not move the
60-second plateau. Raising only
RATE_LIMIT_MAX_REQUESTS=1000000 restored three fresh 3,500-execution runs to
3,814–4,106 workflows/s, median 3,855, with 852–918 ms durations.
Interpretation: the 60-second result is the configured protocol safety cap, not SQLite checkpoint degradation. It is still the correct default-topology behavior and must not be hidden. Capacity campaigns that tune the limit are labelled tuned.
The scale campaign used 5,000 linear executions per instance, one discarded
warm-up, three measured barriers, Embedded concurrency 128, TCP concurrency 64,
and RATE_LIMIT_MAX_REQUESTS=1000000.
| Mode / instances | Median wf/s | p05 / p95 | CV | Speedup | Efficiency |
|---|---|---|---|---|---|
| Embedded ×1 | 3,194 | 3,117 / 3,234 | 1.53% | 1.00× | 100% |
| Embedded ×4 | 10,579 | 10,555 / 10,940 | 1.65% | 3.31× | 82.80% |
| Embedded ×8 | 19,379 | 19,234 / 19,785 | 1.20% | 6.07× | 75.84% |
| Embedded ×12 | 25,873 | 25,293 / 26,114 | 1.34% | 8.10× | 67.50% |
| TCP ×1 | 4,207 | 4,090 / 4,251 | 1.62% | 1.00× | 100% |
| TCP ×4 | 11,738 | 11,627 / 12,324 | 2.57% | 2.79× | 69.75% |
| TCP ×8 | 17,407 | 17,338 / 17,422 | 0.21% | 4.14× | 51.72% |
| TCP ×12 | 17,496 | 17,006 / 17,655 | 1.59% | 4.16× | 34.66% |
At ×12, barrier skew stayed below 1 ms. Embedded peaked near 1,028% sampled CPU and 2,549 MiB RSS. TCP peaked near 2,415% CPU and 3,861 MiB RSS. TCP is effectively saturated by ×8 on this shared host; ×12 consumes more CPU without material throughput gain.
The 24 measured horizontal barriers contain 750,000 exact workflow executions. The eight discarded warm-up barriers add 250,000 more, and each warm-up also passes the same integrity scan before being discarded statistically.
The isolated sandbox on the same runtime revision passed:
| Suite | Pass | Fail | Skip | Anomalies |
|---|---|---|---|---|
| Unit/model | 6,250 | 0 | 3 | Mixed-suite memory growth signal |
| TCP integration | 430 | 0 | 0 | None |
| Embedded integration | 273 | 0 | 0 | None |
The unit suite's end-to-start RSS increase was treated as an investigation signal, not a leak conclusion. Three focused fresh-process TCP chaos soaks then completed:
- 30K jobs and 27 worker kill/reconnect cycles per run;
- 90K total jobs and 81 kill cycles;
- server-authoritative completion reconciliation;
- flat p99 after the initial cold probe;
- WAL around 4.1–4.3 MB;
- post-compaction/post-GC collection bounds passed.
An exploratory SOAK_KILL_MS=300 override failed only its kill-count
expectation: 30 ms reconnect overhead permits 36 rather than the expected 37
kills in 12 seconds. Job conservation and latency passed up to that assertion;
the supported 400 ms profile passed 3/3.
- The highest single-process queue rate is the internal in-memory batched path. It is not an on-disk public API claim.
- For on-disk public producers, the sustained Embedded curve and fresh TCP
PUSHBresult are the relevant figures. - Producer ingestion substantially exceeds TCP worker drain. Capacity planning must size consumers from the ~17K jobs/s single-worker knee, not 159K producer ingestion.
durable:trueis a separate service level and should never be averaged with buffered mode.- A single Workflow Engine sustains roughly 2–3K multi-node no-op workflows/s at the 500–1,000 execution scales measured here.
- Workflow scale-out is useful through 12 Embedded instances. TCP reaches a host-level plateau around 8 pairs and should not be extrapolated linearly.
- The default 10K-request protocol cap is reachable by bursty workflows. Operators should tune it from expected command expansion and retain rate-limit telemetry; benchmarks must state the override.
Queue runners:
bun run src/benchmark/million-jobs.bench.ts
BENCH_N=50000 BENCH_RUNS=21 bun run bench:tcp
bun run bench:pushbulkWorkflow single-engine campaign:
BENCH_MODE=embedded \
BENCH_RUNS=7 BENCH_RUNS_LINEAR=21 BENCH_WARMUPS=1 \
BENCH_N_LINEAR=1000 BENCH_N_PARALLEL=500 \
BENCH_N_COMPENSATION=500 BENCH_N_SIGNAL=500 \
BENCH_CONCURRENCY=128 BENCH_OUTPUT=/tmp/workflow-embedded.json \
bun run bench:workflow
BENCH_MODE=tcp \
BENCH_RUNS=7 BENCH_RUNS_LINEAR=21 BENCH_WARMUPS=1 \
BENCH_N_LINEAR=1000 BENCH_N_PARALLEL=500 \
BENCH_N_COMPENSATION=500 BENCH_N_SIGNAL=500 \
BENCH_CONCURRENCY=64 BENCH_OUTPUT=/tmp/workflow-tcp.json \
bun run bench:workflowTuned horizontal campaign:
RATE_LIMIT_MAX_REQUESTS=1000000 \
BENCH_INSTANCES=1,4,8,12 BENCH_RUNS=3 BENCH_WARMUPS=1 BENCH_N=5000 \
BENCH_CONCURRENCY_EMBEDDED=128 BENCH_CONCURRENCY_TCP=64 \
BENCH_OUTPUT=/tmp/workflow-scale.json \
bun run bench:workflow:scaleRun without BUNQUEUE_EMBEDDED=1; that test-preload variable would force
supposed TCP clients into Embedded mode.
- Results are one native host and one revision, not a hardware-independent SLA.
- Localhost TCP avoids real network delay.
- No-op workflow handlers isolate orchestration overhead; real application handlers can dominate both throughput and latency.
- Horizontal instances share one kernel, scheduler, memory subsystem, loopback stack, and storage device. The campaign identifies aggregate host capacity, not a single contention source.
- Process CPU is sampled and RSS includes runtime/allocator behavior; neither is an exact component allocation.
- Memory slope in a shared test process is not proof of a leak.
Raw per-sample JSON, logs, time output, and resource observations were retained locally under the sandbox artifact tree. All temporary SQLite, WAL, and SHM files and benchmark processes were removed after measurement.