v0.3.0: MQ4R + Redline across RDNA#534
Conversation
Add an opt-in, replay-described gfx1100 QKVZA variant that recovers the explicit shuffle reduction and stable row-stride schedule from the certified autoresearch history. Keep the accepted product default unchanged until exact PM4 and stationary tg128 validation.
Add an opt-in minimum parallel-width floor for phased retained PM4. Narrow antichains now fold back into the serial IB, allowing gfx1100 to retain only overlap that can amortize a cross-queue signal fan-in. The one-queue product default remains unchanged.
Possible gfx1100 MQ4R long-prefill regression in betaThank your effort in putting together this v0.3.0 promotion PR. While validating the headline Qwen3.6-35B-A3B MQ4R path, I came across a result that might indicate a gfx1100 regression in beta and might be useful to consider as part of the final hardware matrix. Environment: 2 × Radeon Pro W7900 ( The matched results were:
The clean-beta result might suggest that my uncommitted local changes are not the primary cause. The MQ4R artifact also has the expected size and digest, which makes weight corruption less likely. Reproduction: HIPFIRE_KV_MODE=q8 HIPFIRE_DPM_WARMUP_SECS=3 ./target/release/examples/bench_qwen35_mq4 ~/.hipfire/models/qwen3.6-35b-a3b.mq4r --prefill 8192 --prefill-runs 1 --warmup 2 --gen 8The 8,192-token prefill completes normally, after which beta fails or stalls at The same boundary also failed after a 32,768-token prefill. Short ordinary AR/MTP inference can still work, so this might be narrower than a general MQ4R failure. The reproducible trigger appears to be the long-prefill → first-single-token transition. In this test series, That commit changes Possible next steps
I would be happy to rerun any additional configuration that might help narrow this down further. |
|
mq4r vs mq3p VRAM A/B on gfx1100 — build flags, results, and repro Ran an isolated A/B measuring load-time VRAM for Environment
Build ( Load config (identical for both models)
VRAM measurement Peak taken as the max over rapid repeated reads during load through warm-up. Results (peak VRAM at warm-up, MTP off, kv=q8)
Per-allocation ledger (via an Reproduce
|
Replay system integration auditInvestigated whether the Redline replay system ( Current stateThe replay controller is completely inert in the daemon's inference path:
Root causeThe replay state machine starts at The replay system was wired into the old Performance impact: zeroMeasured DeepSeek V4 MTP on both branches (clean builds, post-warmup, 3 runs each):
The 36 beta commits (~23K net lines) — including the Redline crates, gfx1151 i8-MMQ prefill path, attention tile tuning, and additional kernels — produce no measurable performance delta for DSv4 daemon-based spec-decode inference on this hardware. The i8-MMQ and tile32 gating conditions (large-batch prefill only) never trigger during MTP decode verify (batch_size ∈ {2,3}). Work needed to activate replay for all model families
This is not a beta-vs-master regression — the infrastructure was added but the integration into the production inference path across arch families was not completed. The PR description correctly lists Redline as an in-tree substrate; the notes above describe the remaining gap for daemon-side activation. |
Docs-only replay of prerequisite lfm-redline documentation changes. No runtime, tooling, root-document, or agent-skill paths are included. Source commits: ac1133a 1378eb8 d999807 e8831ae 95fdccd 7e8c9ca bf9217b 3a025f4 8b031c0 c635e4c 384e0b2 8c43b1e feaed9e 9f28fcb 38d6434 Signed-off-by: Kaden Schutt <151092359+Kaden-Schutt@users.noreply.github.com>
Exact docs/** extraction of parent-to-commit delta from: d335cc9 No non-documentation paths are included. Signed-off-by: Kaden Schutt <151092359+Kaden-Schutt@users.noreply.github.com>
Promote the lfm-redline versions of executable skills already present on beta and add the Redline discovery hook required by docs/REDLINE.md. Source tree: d335cc9 Comparison base: 9ffb18d Intentionally excluded new unrelated skills: agent-memory, gfx-kernel-metadata, and serve-restart. Signed-off-by: Kaden Schutt <151092359+Kaden-Schutt@users.noreply.github.com>
Select the certified gfx1151 fusion, attention, router, and PM4 acquire behavior directly from the exact architecture predicate. Remove the umbrella HIPFIRE_GFX1151_RADIOWAVE_FUSIONS admission variable while retaining the explicit PM4 wait-policy override. No gfx1100, gfx12, or gfx10 predicate changes.\n\nValidated: cargo test -p rdna-compute attention::tests --lib; cargo check -p hipfire-dispatch -p hipfire-arch-qwen35.
Add the exact-gfx1030 kernel, direct Gpu probe method, and scalar-reference channel test without wiring any production model or replay dispatch. The HIP source compiles a real body only for __gfx1030__; other architectures receive an unselected stub.\n\nValidated: cargo check -p rdna-compute --example channel_q8_mmq_gfx1030; hipcc compile for gfx1030/gfx1100/gfx1151/gfx1201; call-site audit shows the method is referenced only by its channel test.
Add the exact-gfx1030 MQ2-Lloyd grouped kernel, a direct probe-only Gpu method, and its scalar/Q8 reference channel test. Do not alter Qwen admission, MoE resolution, public grouped dispatch, replay catalogs, PM4, or shared kernels. The real HIP body compiles only for __gfx1030__.\n\nValidated: cargo check -p rdna-compute --example test_mq2g256_lloyd_moe_grouped_mmq_gfx1030; hipcc compile for gfx1030/gfx1100/gfx1151/gfx1201; call-site audit shows the method is referenced only by its channel test.
Launch the two quarantined gfx1030 channel probes through raw HIP instead of launch_maybe_blob. They remain absent from production dispatch, and an armed Redline recorder cannot absorb either experimental kernel. Existing production launch sites are unchanged.\n\nValidated: cargo check -p rdna-compute --example channel_q8_mmq_gfx1030 --example test_mq2g256_lloyd_moe_grouped_mmq_gfx1030; staged diff contains exactly the two probe methods.
Restore the existing gfx1151 MQ2 grouped method byte-for-byte to beta's launch_maybe_blob path. Route only the new exact-gfx1030 MQ2 probe through raw HIP, matching the already-isolated Q8 probe, so neither gfx1030 experiment can enter Redline capture. Validated: gfx1151 MQ2 method exact-diffs clean against origin/beta; both gfx1030 probe bodies contain exact is_gfx1030 guards and raw launch_kernel with no launch_maybe_blob; cargo checks for both channel examples; 22 Redline replay tests.
Extract the intended delta from PR #532 onto the protected beta-derived branch. Remove the logging-only variant-name table along with the per-dispatch stderr line so the fix does not introduce new unused-variable warnings.
Threshold rejections are an expected control-flow decision and already fall back to WMMA. Keep genuine screening errors visible while removing per-weight unsafe spam.
|
@fivetide — thank you for digging into this. Your gfx1151 result found a real integration gap: One scope correction, because it points toward a useful contribution: There is now a better non-Qwen reference than the Qwen adapter: Useful source landmarks:
Please use that checkpoint as a reference, not a wholesale cherry-pick; it is a large branch checkpoint. If you want to help with DeepSeek, the clean first contribution would be a DeepSeek plain-AR capture boundary on gfx1151, with no automatic-default or speedup claim yet:
A first PR that adds only the plain-AR adapter boundary, negative gates, and a stable/reconciled capture fingerprint would already be valuable. We can review that boundary and help work through the gfx1151 PM4 lowering from there. The immediate goal is not “make the env flag fast”; it is “make one exact DeepSeek forward safely lowerable and observable,” then optimize it deliberately. |
Collapse each uniform MQ4 routed-expert layer from 512 weight BOs into one gate-up owner and one down owner. Preserve the existing per-expert WeightTensor and pointer-table ABI with non-owning subviews, and free metadata and packed owners through their distinct ownership paths. Mixed quant, ParoQuant, paged, and EP-streaming loads retain their prior ownership path. Keep gfx12 on the original per-expert allocations: an all-expert layer BO regressed gfx1201 HipGraph tg128 from 171 to 77 tok/s, while gfx11 dGPU and APU routes validated cleanly. Validation on qwen3.6-35b-a3b.mq4r (SHA-256 4685c140...): - cargo check and 147/147 hipfire-arch-qwen35 library tests pass - gfx1100 GPU allocations: 21,423 -> 1,023; requested bytes unchanged at 18,896,841,988; resident VRAM 23,756,001,280 -> 19,406,700,544 - gfx1100 product: HIP 221.053, PM4 250.248 tok/s, valid - gfx1151 product: HIP 102.323, PM4 115.227 tok/s, valid and above the certified 115.021 floor - gfx1201 guarded product: HIP 171.621, PM4 200.370 tok/s, valid; the same-tip control measured HIP 171.116, PM4 202.375 - exact retained replay passes: gfx1100 604 launches/22 kernels, gfx1151 604/23, gfx1201 733/23 - gfx1100 eight-turn serve session: 8/8 complete, zero runaway/empty/ attractor rows, expected 2/3 recall on turns 7 and 8
|
@ghazni101 — thank you; your 23.79 GB observation was real and led us to the right pressure point. This is now fixed on We reproduced with the same MQ4R artifact digest (
That recovers 4,349,300,736 bytes (~4.35 GB / 18.3%) without dropping any requested tensor payload. The trace narrowed the non-weight delta further than the original cross-format ledger: uniform MQ4R created 20,480 separately owned expert-weight BOs ( Validation on the fixed binary:
The optimization is deliberately gfx11-only. An initial gfx12 A/B exposed a HipGraph regression from the large shared BOs, so gfx12 retains its prior allocation behavior rather than admitting that regression. Please rerun your exact container recipe against |
Follow-up: gfx1100 long-prefill to first-decode failure isolatedThank you for the additional investigation and for fixing the separate MQ4R allocation-count issue in I isolated the latter failure to
This explains why setting I opened PR #538 against
The memory cost is only 8 KiB per scratch instance, so it is unrelated to the multi-gigabyte residency issue fixed in Validation used the same MQ4R artifact digest reported earlier:
There is no performance claim from the single-run on/off timing; the important result is restoring correctness at the reproducible transition boundary while retaining the default-enabled fusion. This PR intentionally does not change the fused-kernel algorithm or dispatch policy: it only corrects scratch capacity and adds pre-launch validation. The resulting VRAM increase is just 8 KiB per scratch instance. |
Verification: commit 0e5762b resolves the ~4 GB mq4r vs mq3p VRAM gapTested on gfx1100 (RX 7900 XTX, 24 GB VRAM, ROCm core-7.14, HIP 7.14). BuildBinary confirmed: 22 MB ELF, Methodology
ResultsWith MTP sidecar loaded (446 MB on disk / ~582 MB in VRAM)
Without MTP sidecar (moved to /tmp, zero MTP lines in log)
Both show the same ~320 MB gap regardless of MTP (MTP adds ~582 MB equally to both models) and regardless of KV cache size (tested both max_seq=512 and max_seq=8192 — identical delta). File size vs VRAM delta
The 1.46 GB file-size delta does not carry through to VRAM because the two quant formats (MQ4 vs MQ3 group schemes, e.g. VerdictThe original gap of ~4,200 MB (measured July 17 on the beta build) is now ~320 MB under commit 0e5762b. The fix for the per-expert 512-BO allocation overhead in |
|
Rust-only control plane landed on What changed:
Validation:
Scope note: this validates the control-plane migration and script wiring. It is not a fresh physical-GPU or retained-PM4 certification, and this commit does not change Redline kernels or the daemon protocol. The Nix expressions were source-reviewed, but Nix tooling was not installed locally for parser evaluation. |
|
Performance and 200K context on gfx1100 (beta@2a960e1) Re-ran the headline Qwen3.6-35B-A3B MQ4R path against the latest beta HEAD. Performance — TG128 after proper thermal soakMeasured through the daemon FIFO protocol after 5x pre-heat generations (GPU at thermal steady state). The daemon's Decode is flat at ~211-213 tok/s across all tested generation lengths (64-8192, sigma < 0.5 tok/s) and prompt lengths (16-1024). First generation after a cold GPU is 5-15% slower — heat soak matters for reproducible numbers. Your 253.3 tok/s reproduces on this card at 212.3 tok/s (16% gap). GPU hits full boost (3139 MHz core, 1249 MHz memory) at 306W draw vs the 339W cap — the remaining gap is card-level power/clock variation, not software. Tested both ROCm 7.14 and 7.2, HIPFIRE_GRAPH=1, and COMPUTE power profile — none moved the needle. 200K contextThe daemon loaded and served with
200K context fits on 24 GiB with ~1 GiB headroom for generation scratch. Decode speed does not degrade with the larger KV allocation — only attention prefill costs increase. |
7 participants
Summary
This draft promotes the accumulated
betastaging line toward hipfire v0.3.0.The release integrates Redline as an in-tree RDNA dispatch and retained-replay substrate, establishes Qwen 3.6 35B-A3B MQ4R as the headline performance SKU, refreshes the model/serving surface, and carries cleanly extracted contributor deltas from the PR backlog.
Contributor work was rebased or cherry-picked as a delta onto the current beta line rather than merged wholesale. That keeps unrelated older code from overwriting the newer modular runtime and Redline implementations.
Headline: MQ4R + Redline
Redline records hipfire's actual decode graph, derives resource dependencies, retains invariant command state, and lowers validated routes through public ROCr queue interfaces. The substrate supports the RDNA family from RDNA1 through RDNA4; optimized routes remain architecture- and workload-specific and fail closed to ordinary HIP dispatch.
Qwen 3.6 35B-A3B MQ4R ordinary AR with Q8 KV:
Short-context ranges were 253.04-253.48 tok/s on gfx1100, 115.02-115.18 tok/s on gfx1151, and 203.42-204.04 tok/s on gfx1201.
These are ordinary autoregressive results: no MTP, DFlash, speculative acceptance, reduced-output benchmark, or manual clock pinning. The gfx1201 campaign raised the path from approximately 110 to 203.9 tok/s.
User-facing release changes
hipfire list -ras the live source of truth.Contributor deltas included
PR #496 remains default-off behind its feature flag; no physical-GPU release claim is made for that experiment.
Additional beta work
redline-dispatchandredline-rocrcrates.Validation
Release metadata commit:
cargo check --workspace --libcargo test --lib --workspace --lockedpython3 scripts/registry_gen.py --checkbun run typecheckbun test: 288 passed, 0 failedgit diff --checkThe registry generator re-probed every curated Hugging Face repository and produced a valid strict superset with 54 models and 38 aliases.
The Rust commands complete with existing unused/dead-code warnings; no new build error was introduced by the release version bump.
Hardware evidence for the headline MQ4R table comes from the recorded beta/Redline product and eight-turn serving artifacts on gfx1100, gfx1151, and gfx1201. This final release-metadata commit did not rerun the physical GPU matrix.
Remaining before promotion