Skip to content

v1.0-performance.md

Latest commit

 

History

History
270 lines (206 loc) · 12.3 KB

v1.0-performance.md

File metadata and controls

270 lines (206 loc) · 12.3 KB
Raw

v1.0 performance journey — how we went from 27 FPS to 60 FPS

This document walks through every perf-relevant change that landed between v0.1.1 and v1.0, why it was needed, and what the measured impact was on the test bench described in v1.0-release-overview.md.

Skip to the section that matters to you:

1. ROV vs RTV

The SDK has two render-target paths for emulating the X360 Xenos GPU's EDRAM:

  • RTV (Render Target Views) — uses native D3D12 render targets plus separate compute passes for resolve / format conversion. The "obvious" port and what early ReXGlue builds defaulted to.
  • ROV (Rasterizer-Ordered Views) — the entire EDRAM is emulated inside the pixel shader via RWByteAddressBuffer writes with ROV ordering. DXBC bytecode is 5-10× larger per shader; cold-compile is slow.

Conventional wisdom said RTV is faster. We measured the opposite on warm cache:

Path Cold-compile Warm-cache FPS Class A fence storms (median per frame)
RTV ~1 second 36 FPS 7
ROV ~13 seconds 58 FPS 0

The ROV path's pixel-shader EDRAM emulation avoids the explicit fence-and-flush dance between draw and resolve that the RTV path needs. Once the shaders are warm the per-frame fence-wait count drops to zero in steady play.

v1.0 ships with render_target_path_d3d12 = 'rov' and a 234 MB pre-built .pso_lib blob (~1,700 entries) so the cold-compile happens once per GPU install, not every launch. The blob is GPU-vendor-specific so we don't distribute it pre-baked — it builds locally on first launch.

The SDK auto-falls back to RTV on GPUs without ROV support (render_target_cache.cpp adapter detection).

2. PSO no-block at submission end

PipelineCache::EndSubmission in the SDK had unconditional logic:

  1. If background PSO compile threads were still busy, wait until they finish, OR
  2. Until d3d12_pso_compile_budget_ms (16.0 ms in our toml) expires.

This fired every frame, even with pso_missing_policy = 'skip'. The skip policy short-circuits IssueDraw for missing PSOs but does NOT skip the end-of-submission wait. Any frame that touched a fresh PSO permutation cost up to 16 ms of render-thread spin at submission close — visible to the user as millisecond stutter.

New cvar d3d12_pso_no_block_at_submission_end (default true) bypasses the wait branch entirely. The trade:

  • Before: per-frame 0-16 ms wait, guaranteed PSO availability by next frame, no missing-geometry artifact.
  • After: 0 ms wait at submission close, PSO ready 1-2 frames later, brief one-frame missing-geometry on FIRST encounter of a new permutation (skip policy handles it).

Net player perception: continuous fluidity beats occasional one-frame pop-ins. Verified in playtest 2026-06-27: user-reported stutters during shader compile in normal gameplay disappeared after this cvar landed.

3. PSO library blob cache

D3D12's ID3D12PipelineLibrary persists driver-compiled PSO microcode to disk. The SDK now writes cache/shaders/local/<title>.<path>.d3d12.pso_lib at session end and reloads it at next session init.

Real measured impact on RTX 5070:

Metric Without library With library
Init wall time 2.72 s 2.36 s (~13% better)
Runtime first-miss sync_compile_ms 32 ms 2 ms (~15× better)

Init time barely moves because the cold-compile already runs across 6 threads. The big win is on RUNTIME first-miss — when a permutation appears mid-gameplay (e.g. first time you see a specific gun firing animation), the driver can re-use its previous compilation work instead of starting from scratch.

Mutex guards Load/Store (D3D12 spec is single-threaded for these). Slow CreateGraphicsPipelineState calls stay outside the lock.

4. Memexport readback

UE3 uses memexport (Xenos's "write to arbitrary VRAM, CPU reads it back" feature) for vertex skinning matrices and HUD text composition. In dense scenes this hits 37+ keys per frame, each historically triggering an AwaitAllQueueOperationsCompletion to ensure the GPU finished writing before the CPU read. That's a fence-storm: 37 sub-millisecond submission waits per frame.

v1.0 ships:

  • Triple-buffer ring (kReadbackBufferSlots = 3) for memexport readback buffers. The fast path reads from "slot written 2 frames ago" which is guaranteed-completed without an explicit fence wait.
  • Targeted per-submission fence wait — when the fast path can't be used (cache miss, partial coverage), it falls back to the slow Full path with a single explicit fence. We tried twice to eliminate that fence entirely (memset zero-init + plain early-return); both attempts TDR'd the GPU because UE3's PPC code reads the returned memory and feeds it into indirect-draw counts / skinning matrices. The CPU MUST have valid post-memexport data.
  • Batched end-of-frame AwaitAll — designed but NOT shipped (TDR risk, needs 2-3h focused refactor session). Deferred to v1.0.1. See v1.0-known-issues.md.

Net result: steady-state gameplay fence_waits dropped from ~18/frame (v0.1.1) to 0/frame. Two known CRITICAL spike events per ~5 minutes of play during scene transitions still hit ~500 memexport readbacks in one frame; those are the deferred Class B work.

5. VFS negative cache

The guest UE3 code probes the same nonexistent paths many times per second (localization fallbacks, temp:\T_* save-backup probes, missing-monolithic- package retries). Each miss otherwise hit HostPathDevice::ResolvePath, which does a case-insensitive ListFiles on the parent directory. Measured: 300-2000 ms frame stalls correlated with [fs] VFS: entry not found log entries.

Fix: in VirtualFileSystem::ResolvePath a small std::unordered_set caches normalized paths that resulted in "not found". The cache is invalidated by RegisterDevice / RegisterSymbolicLink / CreatePath / DeletePath so legitimate file creates aren't masked. Capacity-bounded with drop-on-overflow (cluster around boot + level transitions makes a periodic full reset fine).

Net result: in v1.0 testing, NONE of the 19 spikes >50 ms showed VFS-entry errors. Class B residual is now bounded by other causes (memexport readback, scene streaming) — see known-issues doc.

6. VRAM-aware texture cache

D3D12Provider::Initialize reads DXGI_ADAPTER_DESC.DedicatedVideoMemory and picks a tier from this table:

VRAM available texture_cache_memory_limit_soft _hard
≤ 3 GB 1024 MB 2048 MB
3-5 GB 1536 MB 3072 MB
5-7 GB 2048 MB 4096 MB
7-10 GB 3072 MB 6144 MB
≥ 10 GB 4096 MB 8192 MB

Override is applied ONLY if both cvars are at their SDK defaults (2048/4096) — explicit toml pins win. The canonical v1.0 toml no longer seeds those cvars, so auto-tune fires on every fresh install.

RTX 5070 (12 GB) → enthusiast tier → 4096/8192 MB pools. Tested at 2× supersampling without VRAM-eviction churn.

Per-launch log line: Texture cache auto-tune: GPU has 12288 MB VRAM (enthusiast (>=10 GB VRAM)), set soft=4096 MB hard=8192 MB.

7. PSO no-block-at-submission-end

(Already covered in §2 — search "16 ms stall" above.)

8. Per-frame instrumentation

Without runtime telemetry we'd have been guessing at each of the above. v1.0 includes three layers of per-frame instrumentation that all funnel into the same PSO_STALL[...] log line emitted by PipelineCache::ReportFrameBoundary.

Render-thread buckets (existing)

  • sync_compile_ms — render thread blocked on CreateGraphicsPipelineState
  • block_wait_ms — per-draw spin on async PSO not-yet-ready
  • submit_ms — ExecuteCommandLists time
  • present_ms — IDXGISwapChain::Present time
  • tex_req_ms — texture upload requests
  • rt_resolve_ms — render-target resolve dispatches
  • submit_barriers_ms — resource-state transition barrier counts
  • fence_wait_ms — CPU-side WaitForSingleObject on submission fences

Fence-wait sub-buckets (NEW in v1.0)

Spike fence storms (500+ waits/frame) used to attribute to one generic fence_wait_ms bucket. v1.0 splits it:

  • fence_wait_qops_ms — out-of-submission queue ops (tile mapping etc.)
  • fence_wait_submission_ms — submission_fence_ waits (end-of-frame AND memexport readback Case A funnel here)

Spike CRITICAL frames now clearly show qops=0, submission=525 — confirming memexport readback as the culprit.

Guest-CPU buckets (NEW in v1.0)

Class B residual (unaccounted_ms 200-358 ms with all render-thread metrics at zero) was hypothesized to be guest-CPU stalls. v1.0 ships an RAII scoped timer wrapping the kernel emulation entries:

  • guest_cpu_io_ms / guest_cpu_io_calls — NtCreateFile, NtReadFile, NtWriteFile, NtQueryFullAttributesFile, NtQueryDirectoryFile, NtFlushBuffersFile, NtDeviceIoControlFile, NtReadFileScatter.
  • guest_cpu_alloc_ms / guest_cpu_alloc_calls — RtlAllocateHeap, RtlFreeHeap, RtlReAllocateHeap.
  • guest_cpu_wait_ms / guest_cpu_wait_count — KeWaitForSingleObject, NtWaitForSingleObjectEx, KeWaitForMultipleObjects, NtWaitForMultipleObjectsEx, NtSignalAndWaitForSingleObjectEx.

Each timer is ~30 ns RDTSC pair → negligible overhead even at 5000 allocations/frame.

Drained in ReportFrameBoundary alongside the render-thread buckets. A new label [guest-cpu-stall] is emitted when guest_cpu_total_ms > 5.0 even if no render-thread bucket trips.

unaccounted_ms was redefined: frame_ms - max(render_thread, guest_cpu) instead of just subtracting render-thread. The two run concurrently, so summing them would over-count.

What we learned from the buckets

On v1.0 stable gameplay:

  • Render-thread accounted = sub-millisecond on most frames.
  • Guest-CPU wait ~100-120 ms per frame across ~17 wait calls — this is HEALTHY: audio thread + background loader + render proxy all sleeping on their respective queues at 60 Hz. Doesn't block the actual game thread.
  • guest_cpu_io and guest_cpu_alloc are typically 0 because the VFS cache absorbs misses and the heap is fast.

On spike frames (>50 ms):

  • CRITICAL spikes (>250 ms): submission fence_wait of 120-160 ms across 450-525 waits. ALWAYS clustered on scene/level transitions. Cause is memexport readback flood; fix is the deferred batched AwaitAll.
  • SEVERE/EXTREME spikes (50-150 ms): pure guest_cpu_wait_ms bursts with no fence_wait. UE3 streaming/GC pattern — multiple guest threads block on shared queues briefly. Not directly fixable without UE3 source.

Reading list (memory files / source pointers)

If you want to dig deeper than this document:

  • src/graphics/d3d12/pipeline_cache.{h,cpp} — PipelineStallMetrics, ReportFrameBoundary, EndSubmission.
  • src/graphics/d3d12/command_processor.cpp — AwaitAllQueueOperations Completion, CheckSubmissionFence, the memexport readback fast/slow paths.
  • src/filesystem/virtual_file_system.cpp — ResolvePath with negative cache (lines 272-301).
  • src/ui/d3d12/d3d12_provider.cpp — AutoTuneTextureCacheLimits.
  • include/rex/perf/guest_cpu_stall_metrics.h — the scoped timer + bucket enum used by the kernel hooks.
  • src/kernel/xboxkrnl/xboxkrnl_io.cpp, src/kernel/crt/heap.cpp, src/kernel/xboxkrnl/xboxkrnl_threading.cpp — call sites of the timer.