feat(kvflash): pager serialize/deserialize + QK residency library#466
Open
dusterbloom wants to merge 2 commits into
Open
feat(kvflash): pager serialize/deserialize + QK residency library#466dusterbloom wants to merge 2 commits into
dusterbloom wants to merge 2 commits into
Conversation
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
1 issue found across 5 files
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
This was referenced Jun 30, 2026
Collaborator
Author
|
Re-triggering cubic to review the seeded_n fix (commit 81ed9f7). |
Model-agnostic KV-paging primitives in server/src/common/, consumed by qwen35/qwen35moe, gemma4, and laguna backends. kvflash_pager.h: - serialize(max_chunks)/deserialize v1 format with ledger section (was_resident + qk_score + dtype), enabling pooled-prefix snapshot/restore - critical-chunk pinning + deadlock guard - floor_to_chunk helper - identity_prefix_covers query (preserved for existing consumers) kvflash_qk.h: - cosine QK residency scoring with seeded-fallback + overflow-safe seeded_n (limit defaults to 0, not n_chunks, to prevent OOB reads) Tests: test_kvflash_pager (serde + pinning + partial-serialize + ledger round-trip) and test_kvflash_qk (pure-math scoring). Both CPU-only.
kvflash_qk_chunk_scores read seeded[c] for all c < n_chunks, but the seeded buffer comes from a restored ledger whose chunk count can be smaller than the current n_chunks (pool grew after restore) → out-of-bounds read. Fix: add an explicit seeded_n length parameter; chunks at/above seeded_n fall back to missing_score. Caller (KvFlashTargetQkScorer) now passes seeded_scores_n(). Test test_kvflash_qk gains a phase2b case that constructs a 3-entry seeded buffer over 6 chunks and asserts no OOB. Addresses cubic P1 on kvflash_qk.h seeded fallback.
dusterbloom
force-pushed
the
feat/kvflash-library
branch
from
69d51d1 to
f438230
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Model-agnostic KV-paging primitives in
server/src/common/, already consumed by the qwen35/qwen35moe, gemma4, and laguna backends. This is the reusable foundation extracted from the kvflash snapshot-chain work — lands independently of any backend integration.Why split it out
The full kvflash snapshot chain (~4000 LOC) couples tightly into
qwen35_backend.cpp. The library (pager + QK scoring) does not — it touches onlycommon/headers + tests. Landing it standalone gives every backend a shared, tested KV-paging + residency layer without the integration-review burden.Contents
kvflash_pager.h—serialize(max_chunks)/deserializev1 format with ledger section (was_resident + qk_score + dtype) for pooled-prefix snapshot/restore; critical-chunk pinning + deadlock guard;floor_to_chunk;identity_prefix_covers(preserved for existing consumers).kvflash_qk.h— cosine QK residency scoring with seeded-fallback; overflow-safeseeded_n(limit defaults to 0, notn_chunks, preventing OOB reads when a caller omits the length).Verification
llama.cppsubmodule (no fork dependency).test_kvflash_pagerPASS: serde + pinning + partial-serialize + ledger round-trip.test_kvflash_qkPASS: pure-math scoring incl. seeded-sentinel fallback.dflash_common(main's server code) recompiles —identity_prefix_coverskeeps laguna/gemma4 working.Scope
5 files, +713/−18. No server-backend files, no bench/docs noise. Supersedes the library portions of #429/#445/#430/#446 (those integration slices remain tracked separately).
Follow-up PR (qwen35 pooled snapshot-chain integration) will stack on top of this.