Runtime: extend heap-cap emergency recovery beyond current safe allocation classes

[chrisbbreuer]

Follow-up from #29 after zig-gc b99de49, zig-js 78b553a, and zig-js 45a58ce promoted semantics/oom-one-thread.js and added ArrayBuffer byte-slab recovery.

Current status

The landed recovery paths cover these GC-backed capped-context allocation failures:

• GC cell slab allocation failures retry after Context.collectForAllocationFailure() when the current thread can safely collect.
• no-GIL GC-cell slab recovery has a safepoint-owned current-interpreter path from Runtime: make no-GIL heap-cap recovery safepoint-owned #31; it reuses the abort-safe parallel root-publication collector and retries only after a collection actually sweeps.
• GC-backed ArrayBuffer byte allocation failures retry once after the same recovery hook.
• Budgeted runtime allocations now have a guarded one-shot recovery hook (2d4d3ae0): when the heap cap would be exceeded, the quota allocator can collect safely, reserve again, and continue without scattering ad-hoc retries through object/property code.
• GC-backed object side-store garbage can now be reclaimed by allocation-failure recovery under the heap cap (Context heap_limit_bytes object side-store pressure recovers garbage).
• 72beb646 blocks the generic budget-recovery callback inside GcCellBacking allocator-internal critical sections, so GC-cell chunk OOM fails outward to zig-gc's safe outer Heap.create recovery path instead of self-deadlocking while backing locks are held.
• b3c86930 roots pre-reserved heap-limit catch environments in the interpreter's precise environment root stack, so emergency recovery cannot collect a catch scope needed to convert reserved OutOfMemoryError values into JS catch (e) bindings.
• Single-threaded and GIL-mode contexts can run emergency collection safely when peers are parked/published under the supported scan policy.
• Arena-backed heap caps remain fail-closed and non-reclaimable by design.

That is enough for the PR-249 OOM recovery witness, GC-backed ArrayBuffer byte pressure, ordinary budgeted GC-backed side-store pressure, and the current no-GIL GC-cell emergency-recovery slice, while keeping collection fail-closed when peers could hold unscanned roots or when the current allocation site holds a trace-sensitive lock.

Side-store policy covered so far

Object side-store pressure is now covered by focused tests:

• dropped object side stores are reclaimable by explicit collection, and a second allocation wave can then succeed under the same GC-backed cap;
• allocation-failure recovery now also reclaims dropped side stores automatically in safe single-threaded/GIL-capable contexts;
• live side-store pressure still fails closed as host error.OutOfMemory, proving the cap is enforced when collection cannot free enough budget.

The recovery hook is intentionally guarded, recursion-safe, and disabled inside allocator-internal backing locks rather than a blanket retry. Many no-GIL side-store allocations can happen while object property/element or other trace-sensitive locks are held, while GC tracing may need those same locks.

Remaining perfection work

• Extend abort-safe no-GIL / parallel_js emergency recovery beyond the current safepoint-owned GC-cell path, without scanning unpublishing peers or weakening the fail-closed policy.
• Define finer lock-aware no-GIL collect/retry policies for side stores and other non-cell allocations where the allocation site can prove it does not hold locks the tracer needs.
• Add focused tests for every newly recoverable allocation class; keep fail-closed witnesses for allocation classes that remain unsafe to recover generically.
• Keep arena-backed heap caps fail-closed unless a caller explicitly opts into GC-backed recovery.

This should stay separate from #29/#24: the promoted PR-249 witness is covered, but capped allocation recovery is not yet perfect for every allocation class or no-GIL interleaving.

[chrisbbreuer]

Pushed 45a58ce (fix oom: retry arraybuffer bytes after collect) as the next #30 slice.

What changed:

• Interpreter now has an explicit allocation-recovery callback from Context.
• GC-backed ArrayBuffer byte allocation retries once after Context.collectForAllocationFailure() succeeds.
• Added a focused regression where a 20 MiB ArrayBuffer is dropped under a 32 MiB GC-backed heap cap, then a second 20 MiB ArrayBuffer allocation succeeds only because emergency collection frees the first byte slab.

Validation:

• zig build test with filter: heap_limit_bytes retries ArrayBuffer byte allocation
• zig build test with filter: heap_limit_bytes allocation pressure
• zig build test with filter: ArrayBuffer bytes release when wrapper is collected
• zig build threads-test -Dthreads-case=semantics/oom-one-thread.js --summary all
• git diff --check

I deliberately did not add a generic object side-store allocator retry in this slice. Many object side-store allocations happen while holding property/element locks, while GC tracing takes those same locks. A generic allocator-level recovery hook there could deadlock. That part of #30 needs a lock-aware policy or reserve/headroom design rather than a blind retry wrapper.

[chrisbbreuer]

Docs/policy sync landed in 34134b7 (docs(oom): clarify capped gc recovery policy).

The public docs now describe the current split after 78b553a and 45a58ce:

• arena-backed heap caps remain fail-closed and non-reclaimable;
• GC-backed capped contexts can collect and retry at safe GC cell allocation failures and ArrayBuffer byte slab allocation failures;
• the remaining Runtime: extend heap-cap emergency recovery beyond current safe allocation classes #30 work is explicitly no-GIL root-publication emergency recovery and lock-aware side-store pressure, avoiding a generic side-store retry that could deadlock while object property/element locks are held.

Validation:

• bun run docs:build
• git diff --check

[chrisbbreuer]

Follow-up from the full CI gate on 34134b77: the no-suppression TSan run exposed that allocation-failure recovery was still doing a conservative scan of parked peer native stacks in the serialized/GIL recovery path. That races native wait-frame / park-record publication (semantics/oom-one-thread.js, plus the matching TSan unit witness).

Pushed e179650 to split the scan policy: allocation-failure recovery now scans only the allocating thread native stack and fails closed for live no-GIL peers; parked peer stack scanning is opt-in for the older GIL mid-script path. Local validation:

• zig build test -Dtsan=true -Dtest-filter="heap_limit_bytes lets sibling Threads survive one Thread OOM" --summary all
• zig build threads-test -Dtsan=true -Dthreads-case="semantics/oom-one-thread.js" --summary all

This keeps #30 open: true no-GIL recovery still needs an abort-safe allocation-failure parallel collection path rather than a conservative parked-stack scan.

[chrisbbreuer]

Split out #31 as the first narrow no-GIL emergency-recovery slice.

The key seam is not just "run the parallel collector on OOM": allocation-failure recovery currently has only the Context, while the abort-safe parallel collector needs the current Interpreter and safepoint-like root-publication conditions. Some future side-store allocation sites may also hold locks that GC tracing needs, so a generic allocator-level retry remains intentionally out of scope.

#31 tracks making GC-cell heap-cap recovery attempt the existing abort/no-sweep parallel collector only when the current interpreter can safely act as collector, and otherwise preserve the current fail-closed OOM behavior.

[chrisbbreuer]

#31 slice landed in 8db58b6 (fix oom: recover no-gil cell allocation at safepoints).

Parent status update:

• GC cell allocation-failure recovery now has a current-interpreter/safepoint-owned no-GIL path.
• The path reuses the abort/no-sweep parallel collector and only retries allocation after the collection actually sweeps.
• Peers publish for an emergency collector even when routine parallel_midscript_gc is disabled.
• A trace-sensitive lock-depth guard keeps object/environment/promise/iterator side-store locks as fail-closed OOM sites, so this does not introduce the generic side-store retry Runtime: extend heap-cap emergency recovery beyond current safe allocation classes #30 deliberately avoided.

Focused validation included the new heap-cap cell witness under TSan plus the existing ArrayBuffer recovery and mid-script parallel collector witnesses. Keeping #30 open for the remaining lock-aware side-store pressure policy and broader allocation-class coverage.

[chrisbbreuer]

Docs sync landed in c972087 (docs oom: sync no-gil heap cap recovery status).

docs/threads/limits.md now reflects the current #30/#31 split:

• GC-backed caps can retry safe GC-cell and ArrayBuffer byte allocation failures.
• no-GIL GC-cell slab recovery is safepoint-owned by the active interpreter and uses the abort-safe parallel root-publication collector.
• recovery still fails closed outside active JS execution or while holding side-store locks the tracer may need.
• Runtime: extend heap-cap emergency recovery beyond current safe allocation classes #30 remains open for lock-aware side-store pressure coverage.

[chrisbbreuer]

Pushed 8e875900 (test: cover capped object side-store pressure) as a coverage/policy slice for the remaining side-store part of #30.

What changed:

• Added a capped-context regression showing object side-store pressure currently fails closed as host error.OutOfMemory rather than pretending to be a safe generic retry/catchable path.
• Added a GC-backed capped-context regression showing object side stores from dropped references are reclaimed by explicit collection, then a second allocation wave succeeds under the same cap.

This intentionally does not add a generic side-store allocation retry. The fail-closed test documents the current lock-aware policy: side-store allocations can happen while locks that tracing also needs are held, so recovery there must not blindly collect/retry.

Focused validation:

• zig build test -Dtest-filter="object side-store pressure fails closed" --summary all
• zig build test -Dtest-filter="object side stores reclaim after explicit collection" --summary all
• git diff --check

#30 remains open for a future positive lock-aware recovery mechanism, but the missing side-store pressure/reclaim coverage is now in place.

[chrisbbreuer]

Follow-up CI for 8e875900 was superseded by the #34 push, and the combined head 919f148ba90a3c88b78d16d8447e5945374f71c1 passed the full CI gate:

https://github.com/zig-utils/zig-js/actions/runs/29175661429

So the #30 side-store pressure coverage slice is green in the full suite.