Canonical: JavaScript multithreading support

[chrisbbreuer]

Goal

Ship production-quality JavaScript multithreading in zig-js across both isolation models:

• isolated agents/Workers that exchange structured-clone values and retained SharedArrayBuffer storage;
• shared-realm Threads that preserve JavaScript object identity while executing on real OS threads.

This issue is the concise umbrella tracker. Detailed acceptance criteria and implementation work live in the child issues below; commit-level history stays in comments and docs/threads.

Pre-stabilization backward compatibility is intentionally not a constraint; remove accidental compatibility shims when a cleaner public contract is available.

Public Contract

// True-parallel shared-realm JavaScript by default.
const parallel = try js.Context.createWith(gpa, .{ .enable_threads = true });

// Supported serialized fallback.
const serialized = try js.Context.createWith(gpa, .{
    .enable_threads = true,
    .gil = true,
});

The C API exposes the same choice through ZJSGlobalContextCreateThreaded(gil).

parallel_js, parallel_midscript_gc, and related harness/profile controls are internal/test-only. They are not stable embedder APIs.

Implemented

• Real OS-thread agents and Workers, structured clone, ArrayBuffer transfer/detach, retained shared-buffer storage, and typed-array Atomics.wait / notify / waitAsync.
• Shared-realm Thread, Lock, Condition, ThreadLocal, ConcurrentAccessError, property-mode Atomics.*, and proposal-aligned Atomics.Mutex / Atomics.Condition entry points.
• No-GIL execution by default, with dedicated synchronization for shapes, properties, elements, environments, promises, microtasks, inline caches, waiters, thread records, host queues, C-API roots, and GC roots/barriers.
• GC-managed thread-safe cells, precise VM roots, conservative native-stack roots where applicable, abort-safe mid-script parallel full collection, weak collections, and FinalizationRegistry cleanup.
• A demand-driven, quiescent, non-moving one-cycle nursery with owner-aware remembered old containers, weak/ephemeron processing, immediate survivor tenuring, old-space full fallback, and pause/promotion/threshold-drift profiling. Explicit GC remains full-heap; parallel mid-script minor collection is not claimed.
• Heap caps, OutOfMemoryError completion for capped shared-realm Threads, allocation-site catchability for supported active handlers, and GC-backed collect/retry recovery for safe cell and ArrayBuffer byte allocation pressure. Arena-backed caps remain fail-closed; broader no-GIL emergency recovery and lock-aware side-store/non-cell pressure remain Runtime: extend heap-cap emergency recovery beyond current safe allocation classes #30. Deferred generator/iterator storage is intentionally all-fail-closed under the closed Runtime: prove generator and iterator side-store heap-cap recovery under no-GIL #36 policy.
• Documented separation between JavaScript program races and engine-state races, with ThreadSanitizer currently suppression-free.
• Security/resource-control hardening for harness path confinement, diagnostic fixture loading, shared-buffer retain overflow, protected C-handle count overflow, wrong-context C handles, worker-handle affinity, ArrayBuffer/DataView resize races, and non-deterministic Math.random seeding.

Verification

• Unit and C-API suites cover the public thread modes and synchronization APIs.
• WebKit PR-249 coverage is 236 / 259 executable files: 234 in the default corpus plus 2 no-GIL-only witnesses.
• threads-reference-audit rejects missing blocker classifications and maintains negative promotion probes; the current machine audit reports 23 executable reference-only files plus 5 helper/preload files.
• CI gates unit/corpus behavior, no-GIL TSan, a sharded PR-249 TSan corpus sweep, suppression-free TSan policy, test262-parallel, deterministic concurrent fuzzing, specialized mid-script-GC/lifecycle fuzzing, and longer nightly sanitizer sweeps.
• Focused profilers (threads-profile, midgc-profile, gc-profile) provide measured local evidence before contention, GC, nursery, or lifecycle policy changes are promoted.

Remaining Roadmap

Tracker	Scope
#30	Heap-cap emergency recovery beyond current safe allocation classes
#16	Nursery sizing/depth, GC pause tuning, and Context lifecycle performance
#15	Profile-guided no-GIL contention and scaling
#14	Mid-script parallel GC maturity and sync-wait convergence
#13	Concurrent lifecycle, exception, termination, waiter, and cleanup fuzzing
#12	Shared-realm JavaScript memory-model maintenance and litmus tests
#11	Promotion of remaining WebKit PR-249 reference cases

The remaining PR-249 files depend on real missing or intentionally non-portable behavior such as WebAssembly construction/grow hooks, JIT or shell controls, JSC heap/snapshot hooks, JSC butterfly-verification assumptions, detached-source fresh-view construction assumptions that zig-js deliberately rejects, or semantic cases with current audit evidence. A case is promoted only when the underlying engine behavior exists and focused tests accompany it.

Completion Criteria

• The final public Zig and C semantics are stable, documented, and free of accidental compatibility shims before stabilization.
• No known engine-state races remain in supported paths.
• GC/lifecycle and no-GIL scaling have measured, documented production guidance.
• Mid-script collection has a supportable convergence policy rather than relying on test-only controls.
• Fuzzer and corpus gates cover the supported lifecycle surface at sustained depth.
• Every executable PR-249 file is either promoted or has a current, auditable feature blocker.

Authoritative Docs

• docs/threads/index.md - model and support matrix
• docs/threads/api.md - public shared-realm API
• docs/threads/memory-model.md - race and synchronization contract
• docs/threads/testing.md - local and CI gates
• docs/threads/production-readiness.md - current hardening status
• docs/threads/limits.md - unsupported surfaces and roadmap
• docs/threads/bindings.md - mutable-state synchronization audit

[chrisbbreuer]

Also should keep an eye on this one oven-sh/WebKit#249 - maybe we can reuse quite a bit of it

[chrisbbreuer]

Progress 2026-06-10: Phases 0–2 are implemented and on main (through d85c080).

• P0: Context thread-affinity guardrails (debug asserts, C-API thread rules), docs/threads/bindings.md audit (10 globals ruled; found two the plan missed: Math.random's process-global PRNG and the global symbol counter — both fixed in P2), PR-249 corpus vendored under reference/webkit-249/ (240 test files + full docs/threads specs + THREAD.md, head 25375a99).
• P1: src/shared_buffer.zig — refcounted process-wide SAB storage, in-place CAS grow, RetainList per realm; ArrayBufferData reworked behind bytes() (compiler-forced audit of all ~56 byte-access sites); Atomics RMW/CAS/load/store are single SeqCst hardware ops (raw-bits path also fixed BigInt64 precision); 8-byte-aligned buffers.
• P2: src/agent.zig — $262.agent on real OS threads with the INTERPRETING.md broadcast rendezvous; Atomics.wait genuinely blocks (FIFO waiter table, per-ticket condition + timeout, [[CanBlock]]); notify really wakes; real sleep/monotonicNow; the cooperative model is deleted. Atomics-subtree workers run one test per process so a deadlock costs one 30s timeout.

Score: 41,898 → 42,137/47,928 (87.9%); built-ins/Atomics 308 → 349/388 (89.9%), zero host-fails (no hangs in two full runs). End-to-end concurrency is unit-tested (agent parks → broadcast → blocked wait → cross-thread notify → report).

Remaining Atomics failures are mostly Atomics.waitAsync (101 test files) → that's Phase 3, designed in docs/threads/P2-agents.md (async tickets on the same waiter table + drain-loop integration in Context.evaluate). Also still open from P2: the CanBlockIsFalse runner mode (2 tests).

[chrisbbreuer]

Phase 3 landed (9999301, 28baab2): Atomics.waitAsync settles for real — async tickets share the blocking waiters' FIFO lists (spec notify ordering across wait/waitAsync), and the realm's drain tail resolves the promises on notify/deadline/teardown, abandoning unsettleable infinite waits instead of hanging. The CanBlockIsFalse runner mode closes the last Phase 2 box (both tests now scored and passing).

42,230/47,930 (88.1%); built-ins/Atomics 356/390 (91.3%), still zero host-fails. ~34 Atomics failures remain (under diagnosis). Next big block: Phase 4 structured clone.

[chrisbbreuer]

Phase 4 landed (ba41807): structured clone. src/structured_clone.zig serializes value graphs into a self-contained byte stream (deliberately so — it's the Phase 5 postMessage wire format) and deserializes into any realm's arena. Full type coverage incl. cycles/identity, array holes, SAB storage sharing, and { transfer } (move + detach); DataCloneError for the unclonables. The structuredClone global is installed. test262 unchanged at 42,230/47,930 (88.1%) — additive feature, zero regressions, gate run confirms.

Still open from Phase 4: ArrayBuffer.prototype.transfer predates this work (already done). Next: Phase 5 — embedder Worker API (spawn = Context-per-thread + this wire format + the agent group's channel machinery), then Phase 6 (GIL Thread API, port the vendored PR-249 corpus).