Skip to content

feat: Dash table cache-mode segment-local eviction (#285 stage 2)#486

Merged
ELares merged 2 commits into
mainfrom
feat/dashtable-cache-eviction
Jul 2, 2026
Merged

feat: Dash table cache-mode segment-local eviction (#285 stage 2)#486
ELares merged 2 commits into
mainfrom
feat/dashtable-cache-eviction

Conversation

@ELares

@ELares ELares commented Jul 2, 2026

Copy link
Copy Markdown
Owner

What

Implements DASHTABLE.md stage 2 on the standalone ironcache-dashtable crate: cache-mode segment-local eviction, the (b) lever of #285 (O(1) eviction).

insert_cache(key, value, freq_of) replaces split-on-full with EVICT-on-full: when the routed segment is at capacity, it evicts the coldest slot in that segment and places the new key — so memory stays bounded and eviction touches O(SEGMENT_CAP) = O(1) slots with no table-wide scan and no per-key side state (no evict_pool, no refill_evict_pool amortization).

Same victim as the store — the stage-2 acceptance

DASHTABLE.md stage 2 calls for exactly "a model test asserting the same victim as the current freq-in-object selection on shared inputs." The victim is the slot minimizing the total order (freq, scan_hash, key) — byte-identical to the store's refill_evict_pool ColdEntry order (freq, scan_h, key[, db]; EVICTION.md, ADR-0003). The 2-bit frequency lives in the value (freq-in-object), read via the caller's freq_of accessor, so the table stays generic with zero eviction metadata of its own.

cache_evicts_the_freq_in_object_victim fills a segment, independently recomputes the expected victim via that exact order (an oracle mirroring the store), and asserts insert_cache evicts precisely that key. Plus:

  • cache_higher_freq_survives_eviction — a hot slot (freq 3) is never the victim across 100 evictions.
  • cache_eviction_is_segment_local — with 16 distinct segments, hammering segment 0 leaves every other segment's contents untouched (the O(1) locality claim).
  • cache_insert_overwrites_in_place_without_eviction.

Honest scope

  • This PR proves the eviction VICTIM QUALITY on any host: the O(1) segment-local choice equals the store's current O(N/CAP)-amortized selection, deterministically.
  • Not in scope (later stages, correctly gated): wiring behind the dashtable feature flag (stage 3), and the memory/throughput win vs DragonflyDB — which needs the pinned-Linux + bench harness (stage 4). I am not claiming the perf win here; only the correctness primitive it rests on.
  • Still 100% safe (miri-trivial) and standalone (zero store blast radius).

Tests

7 (3 stage-1 + 4 stage-2) pass; clippy (pedantic, -D warnings) + fmt + invariant lints clean; 0 dashes.

🤖 Generated with Claude Code

DASHTABLE.md's stage 2: the standalone ironcache-dashtable gains cache-mode
eviction, the (b) lever of #285 (O(1) segment-local eviction).

`insert_cache(key, value, freq_of)` replaces split-on-full with
EVICT-on-full: when the routed segment is at capacity, it evicts the coldest
slot IN THAT SEGMENT and places the new key, so memory stays bounded and
eviction touches O(SEGMENT_CAP) = O(1) slots with NO table-wide scan and NO
per-key side state (no evict_pool, no refill bookkeeping).

The victim is the slot minimizing the deterministic total order
(freq, scan_hash, key) -- the SAME freq-in-object order the store's
refill_evict_pool ColdEntry uses (freq, scan_h, key[, db]; EVICTION.md,
ADR-0003), so two shards with identical state evict identically. The 2-bit
frequency lives IN the value (freq-in-object), read via the caller's
`freq_of` accessor, so the table stays generic and carries no eviction
metadata of its own. `with_directory_bits` pre-sizes the segment array for a
known working set (a cache is sized up front; growth is by eviction, not
splits).

The stage-2 deliverable DASHTABLE.md calls for is "a model test asserting
the same victim as the current freq-in-object selection on shared inputs" --
implemented as `cache_evicts_the_freq_in_object_victim`, which fills a
segment, independently computes the expected victim via the store's exact
(freq, scan_hash, key) order, and asserts insert_cache evicts precisely
that key. Plus tests for freq-protects-the-hot-key, segment-locality (16
distinct segments: evicting in one leaves the others untouched), and
overwrite-in-place-without-eviction.

Still 100% safe (miri-trivial) and standalone -- wiring behind the
`dashtable` feature flag is stage 3, and the memory/throughput WIN vs
DragonflyDB is the pinned-Linux head-to-head (stage 4). What THIS PR proves,
on any host, is the eviction VICTIM QUALITY: the O(1) segment-local choice is
byte-identical to the store's current O(N/CAP)-amortized selection.

7 tests (3 stage-1 + 4 stage-2) pass; clippy (pedantic) + fmt + invariant
lints clean; 0 dashes.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: ELares <zeke@butlr.io>
@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown

perf-gate (A5)

Same-runner ratchet of HEAD against the merge-base (both rebuilt and measured in this job).
PASS = within the noise band, WARN = a real move inside budget (does not fail), FAIL = past budget in the bad direction.

metric base head delta% band budget verdict
qps_median (peak) 85537.48 88415.35 3.36% +/-6.57% drop <= 15% PASS
bytes_per_key int 45.02 45.02 0.00% det rise <= 5% PASS
bytes_per_key embstr 61.07 61.07 0.00% det rise <= 5% PASS
bytes_per_key raw 333.15 333.15 0.00% det rise <= 5% PASS

Overall: PASS

  • qps: noisy on shared CI, so the band comes from the base reps spread (floored at 5%); a drop is only a regression past the 15% budget.
  • bytes_per_key: deterministic (allocator-true memmodel), so a tight 5% rise budget; any rise beyond it FAILs.
  • Open-loop tails / criterion micro-benches are reported-not-failed (tail noise is high) and are not part of this ratchet.
  • An intentional perf trade is landed by raising the relevant budget in this PR with a documented reason (CI never auto-commits a baseline).

…order (#285)

The adversarial review found the doc claim "identical to the store's
refill_evict_pool victim order" was OVERSTATED: the tie-break used this
crate's internal directory hash (hash_key, std DefaultHasher / SipHash),
while the store's ColdEntry breaks freq ties with `scan_hash` -- a DIFFERENT
function (a wyhash/FNV-1a fold + splitmix64 finalizer over the raw key
bytes). Among equal-freq keys (the common case in a cold tail) the two would
pick DIFFERENT victims, and the in-crate oracle reused hash_key, so the model
test was self-consistent but blind to the divergence from the real store.

Fix: the scan-order hash is now a CALLER-supplied accessor
(`scan_hash_of: Fn(&K) -> u64`), parallel to `freq_of`. The store passes its
own `scan_hash` when it wires this table (stage 3), so the victim is
byte-identical to refill_evict_pool BY CONSTRUCTION -- not merely
same-shaped. The generic table carries no hash-family assumption; passing the
hash in is exactly what makes the parity real.

The model test now ports the store's ACTUAL scan_hash (byte-exact from
store/src/lib.rs) and feeds it as the accessor, so
cache_evicts_the_freq_in_object_victim validates parity with the REAL store
order, not the crate's internal one. The module + method docs are corrected
to describe the caller-supplied scan_hash and the by-construction parity.

7 tests pass; clippy (pedantic) + fmt + invariant lints clean; 0 dashes.

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Signed-off-by: ELares <zeke@butlr.io>
@ELares

ELares commented Jul 2, 2026

Copy link
Copy Markdown
Owner Author

Deep self-review: adversarial multi-agent workflow

Ran a 3-dimension adversarial review (eviction-correctness / edge-cases / API) with a per-finding skeptic-verify phase. It surfaced one genuine, sharp finding (reported by two dimensions) — now fixed — plus correctly refuted the rest (the min_by-returns-coldest logic, the omitted db tiebreak being right for a single-table crate, evict_victim's non-empty precondition, the with_directory_bits bound, and the len bookkeeping were all confirmed correct).

The real finding: the "identical to the store" claim was overstated

My docs claimed the victim order was "identical to the store's refill_evict_pool victim order". But the tie-break used this crate's internal directory hash (hash_key = DefaultHasher/SipHash), while the store's ColdEntry breaks freq ties with scan_hash — a different function (wyhash/FNV-1a fold + splitmix64 over the raw key bytes). Among equal-freq keys (the common case in a cold tail), the two would pick different victims. Worse, my in-crate oracle reused hash_key, so the model test was self-consistent but blind to the divergence from the real store — defeating the crate's entire stated purpose as a parity reference.

Fix: make the parity real, not just same-shaped

Rather than soften the doc, I made the claim true. The scan-order hash is now a caller-supplied accessor (scan_hash_of: Fn(&K) -> u64), parallel to freq_of. The store passes its own scan_hash when it wires this table (stage 3), so the victim is byte-identical to refill_evict_pool by construction. The model test now ports the store's actual scan_hash (byte-exact) and feeds it as the accessor, so cache_evicts_the_freq_in_object_victim validates parity with the real store order. Docs corrected.

This is the better outcome: the generic table carries no hash-family assumption, and passing the hash in is exactly what makes stage-3 wiring correct.

Post-fix: 7 tests pass; clippy (pedantic, -D warnings) + fmt + invariant lints clean; 0 dashes. Re-running CI.

@ELares ELares merged commit 7d1c396 into main Jul 2, 2026
16 checks passed
@ELares ELares deleted the feat/dashtable-cache-eviction branch July 2, 2026 03:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants