Skip to content

implementation_stages.md

Latest commit

 

History

History
609 lines (559 loc) · 29.5 KB

implementation_stages.md

File metadata and controls

609 lines (559 loc) · 29.5 KB
Raw

Implementation Stages

Kairo uses two patch tracks:

  • Broad RFC/POC architecture patches: concept coverage across the full nine-patch series.
  • Foundation patches: the smaller compile-targeted kernel core under kernel/patches/foundation/.

The foundation stack currently covers Stage 1 and Stage 2 only.

Stage 1

  • Broad RFC/POC patches involved: 0002, 0001, 0009
  • Foundation patches involved: foundation/0001, foundation/0002, foundation/0004
  • What should compile:
    • local request classification helpers
    • mq-deadline decode-priority path
    • aligned Kairo sysfs counters
  • What should be measurable:
    • decode_avg_us
    • decode_p95_us
    • decode_p99_us
    • kairo_decode_dispatches
    • kairo_normal_dispatches
  • What is still RFC-only:
    • broader hint plumbing beyond ioprio
    • anything outside the foundation stack

Stage 2

  • Broad RFC/POC patches involved: 0005
  • Foundation patches involved: foundation/0003, foundation/0004
  • What should compile:
    • prefetch metadata fields and scheduler recognition hooks
  • What should be measurable:
    • prefetch pressure runs versus decode tail latency
  • What is still RFC-only:
    • tuned deadline policy and starvation tradeoff validation

Stage 3

  • Broad RFC/POC patches involved: 0004, 0009, 0002
  • What should compile:
    • Kairo merge-bias helpers (kairo_should_bias_merge, kairo_merge_within_limits)
    • per-request merge-instrumentation flags set in attempt_merge and blk_mq_bio_merge
    • request-size histogram counters consumed at dispatch time
  • What should be measurable:
    • kairo_merge_attempts / kairo_merge_successes / kairo_merge_rejects
    • kairo_decode_merge_attempts / kairo_decode_merge_successes
    • kairo_prefetch_merge_attempts / kairo_prefetch_merge_successes
    • kairo_small_decode_reads / kairo_large_decode_reads (threshold via kairo_large_read_kb)
    • full request-size histogram: kairo_{decode,prefetch}_read_{4k,16k,64k,256k,1m,4m,gt4m}
    • benchmark access patterns: random, sequential, strided, clustered
    • benchmark modes: merge-friendly (sequential, large block), merge-hostile (fragmented, random/session-interleaved)
  • What is still RFC-only:
    • full validation of merge policy on real devices
    • whether the histogram counters are better served by debugfs snapshots instead of sysfs

Stage 4

  • Broad RFC/POC patches involved: 0003
  • What should compile:
    • experimental RWF_KAIRO_* and kiocb plumbing
    • conceptual kiocb -> bio -> request Kairo metadata helpers
  • What should be measurable:
    • benchmark --hint-mode ioprio|rwf|both
    • rwf_*_{attempts,fail} counters in the benchmark summary
    • staged hint-source counters such as kairo_ioprio_hinted_requests
  • What is still RFC-only:
    • scaffolded / local RFC, not compile-validated
    • final local interface choice for hint propagation

Stage 5

  • Broad RFC/POC patches involved: 0006
  • What should compile:
    • ephemeral and recomputable flag scaffolding
    • local semantic flag helpers on kiocb, bio, and request metadata
  • What should be measurable:
    • benchmark --semantic-mode normal|ephemeral|recomputable|ephemeral-recomputable
    • rwf_ephemeral_*, rwf_recompute_*, rwf_no_durability_*, rwf_avoid_pagecache_*
    • semantic counter deltas such as kairo_ephemeral_requests
  • What is still RFC-only:
    • scaffolded / local RFC, not compile-validated
    • exact durability and cache-management semantics

Stage 6

  • Broad RFC/POC patches involved: 0007, 0009
  • Userspace header: include/kairo_hints.h (placement structs and flags)
  • Benchmark: kairo_bench.c (placement CLI options and output fields)
  • Scripts: scripts/run_stage6_placement_experiment.sh, scripts/parse_stage6_placement_summary.py
  • What should compile:
    • enum kairo_lifetime_class, struct kairo_placement_hint, helpers, synthetic default init (blk_mq_kairo_default_placement)
    • scaffold placement/lifetime counters in mq-deadline.c
  • What should be measurable:
    • software-only grouping experiments by model/session/cache pool
    • benchmark --lifetime, --recompute-ok, --cache-pool-id, --placement-group, --cache-pools, --placement-groups
    • sysfs scaffold counters: kairo_placement_hints, kairo_lifetime_*_count, kairo_has_*_count
  • What is still RFC-only:
    • stable mapping semantics through the stack
    • NVMe Streams/FDP/ZNS mapping (deferred to Stage 7)
    • scheduler policy changes based on lifetime class

Stage 6.5

  • Status: benchmark/experiment harness, not foundation-integrated
  • Scripts involved:
    • scripts/run_stage6_placement_experiment.sh — hardened harness accepting <file-path> <block-device> with structured results/stage6/<timestamp>/ output
    • scripts/parse_stage6_placement_summary.py — CSV and pretty-printed summary parser supporting counter delta columns
    • scripts/collect_kairo_counters.sh — Stage 6 counter coverage (both naming sets)
  • What is measurable:
    • five canonical placement/lifetime cases with before/after counter deltas
    • structured run metadata and per-case summary.log files
    • aggregated summary.csv across all cases
  • What is still RFC-only:
    • NVMe/FDP/ZNS mapping (deferred to Stage 7)
    • physical placement control based on placement-group hints

Stage 7

  • Broad RFC/POC patches involved: 0008, 0009
  • Userspace header: include/kairo_hints.h (enum kairo_backend_mode)
  • Benchmark: kairo_bench.c (--backend-mode option, mapping output fields)
  • Scripts: scripts/run_stage7_backend_mapping_experiment.sh, scripts/parse_stage7_backend_summary.py
  • Docs: docs/stage7_generic_nvme_backend_mapping.md
  • What should compile:
    • enum kairo_backend_class, struct kairo_backend_hint, helpers
    • feature-detected NVMe mapping hooks (no-op fallback when absent)
    • backend mapping scaffold counters in mq-deadline.c
  • What should be measurable:
    • backend class mapping from Stage 6 lifetime metadata
    • benchmark --backend-mode none|generic|streams|fdp|zns
    • sysfs scaffold counters: kairo_backend_mapping_attempts, kairo_backend_*_hints, kairo_backend_*_lived/_local/_persistent
    • benchmark output fields: backend_mode, backend_class, stream_id, fdp_placement_id, zone_hint, backend_noop_fallback
  • What is still RFC-only:
    • real NVMe hardware placement (Streams, FDP, ZNS)
    • physical backend mapping effectiveness on target devices
    • detection of NVMe feature bits via identify commands
    • application of backend hints to NVMe commands

Stage 7.5

  • Broad RFC/POC patches involved: 0008 (rewritten sections A-H)
  • Audit doc: docs/stage7_5_nvme_hook_audit.md
  • Audit script: kernel/integration/linux-6.8/audit_nvme_hooks.sh
  • Validator: scripts/validate_stage7_backend_mapping.py
  • What changed:
    • 0008 reorganized into sections A–H with struct kairo_backend_caps abstraction
    • Per-feature nvme_kairo_streams_supported() / _fdp_ / _zns_ replaced by unified nvme_kairo_get_backend_caps()
    • Added kairo_backend_hint_apply_caps() helper
    • backend hint stored inline in struct kairo_backend_hint inside kairo_hints.placement
    • Compile-risk annotations added per hook point (COMPILE-TARGET, CONCEPTUAL-HOOK, VERSION-SENSITIVE)
    • Benchmark refactored: kairo_compute_backend_model() replaces 5 individual helpers
    • validate_patch_stack.sh integrated with Python validator
  • What should be validated:
    • All sections A–H present in 0008
    • Compile-risk annotations match audit document classifications
    • Python validator passes against current 0008, docs, and benchmark
    • Audit script detects candidate symbols in real Linux 6.8 tree
    • Benchmark compiles with refactored backend model computation
  • What is still RFC-only:
    • Real NVMe hardware placement (Streams, FDP, ZNS)
    • Wiring of caps detection to real identify-command data
    • Physical backend mapping effectiveness on target devices

Supernova Patches (0010–0016)

Seven patches in dependency order that harden the Kairo proof of concept from instrumentation-grade to a structurally sound I/O scheduling framework:

Patch Area Status KV perf impact
0010 real classification implemented root dependency — without it, 0001 measures noise
0011 write anti-starvation implemented correctness gate; benchmarks invalid without it
0012 tag reservation implemented highest single-patch latency impact
0013 O(1) dispatch FIFO implemented eliminates latency cliff under contention
0014 io_uring SQE hint implemented enables production-realistic classification
0015 real merge bias implemented reduces NVMe command count per decode step
0016 BPF dispatch hook implemented architectural differentiator: programmable I/O scheduling

Stage 8 (Tracepoints)

  • Broad RFC/POC patches involved: 0017
  • Userspace header: include/trace/events/kairo.h (new)
  • What should compile:
    • tracepoint header include/trace/events/kairo.h (TRACE_EVENT definitions)
    • tracepoint call sites in block/blk-mq.c, block/mq-deadline.c, block/blk-merge.c, drivers/nvme/host/core.c
    • runtime validation scripts
    • bpftrace scripts
  • What should be measurable:
    • per-request lifecycle tracepoints (classification, scheduler decision, dispatch, demotion, merge, semantic, placement, backend mapping)
    • structured trace logs in results/stage8/<timestamp>/
    • trace log parsing with CSV and pretty-printed summary
    • A/B decode latency with tracepoint confirmation
  • What is still RFC-only:
    • all tracepoints (not stable ABI)
    • bpftrace script compatibility across kernel versions
    • end-to-end proof for the full ten-patch series

Stage 9

  • Status: tooling/validation layer
  • Scripts involved:
    • scripts/check_wsl_environment.sh
    • scripts/run_wsl_validation_snapshot.sh
    • scripts/parse_validation_snapshot.py
  • What should validate:
    • WSL-friendly repo validation
    • benchmark build
    • dry-run experiment validation
    • smoke benchmark execution
    • generated validation snapshot evidence packaging
  • What is still RFC-only:
    • custom kernel boot validation
    • patched-kernel runtime counters
    • physical NVMe placement claims

Stage 10

  • Broad RFC/POC patches involved: 0018
  • Docs: docs/stage10_adaptive_latency_controller.md
  • Scripts:
    • scripts/run_stage10_latency_controller_experiment.sh
    • scripts/parse_stage10_latency_controller_summary.py
  • What should compile:
    • enum kairo_controller_mode and struct kairo_latency_controller in mq-deadline.c
    • controller state init, sample accumulation, p95/p99 computation, budget adjustment logic
    • sysfs knobs for controller enable, mode, target p99, control window
    • sysfs counters for controller updates, boost/relax/throttle/release events
    • integration call site in dd_dispatch_request()
    • decode pressure signal in demotion helpers
  • What should be measurable:
    • controller counters in sysfs when controller is enabled
    • kairo_adaptive_decode_budget and kairo_adaptive_prefetch_budget reflect controller decisions after a window of decode pressure
    • six canonical experiment cases comparing static vs controller-driven scheduling
    • structured output under results/stage10/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
  • What is still RFC-only:
    • real decode latency measurement from inside the kernel (completion hook)
    • controller behavior on real NVMe hardware under true inference patterns
    • precise p95/p99 latency histogram (uses avg/max heuristic)
    • timer-based controller update (currently called from dispatch path)
    • kairo_controller_update tracepoint (documented but not wired)
    • interaction with BPF dispatch hook (patch 0016)
    • per-device budget tracking (currently updates global variables)

Stage 11

  • Foundation patches involved: foundation/0005
  • Broad RFC/POC patches involved: 0022
  • Docs: docs/stage11_foundation_tracepoints.md
  • Scripts:
    • kernel/integration/linux-6.8/validate_foundation_tracepoints.sh
    • scripts/run_stage11_foundation_trace_experiment.sh
    • scripts/parse_stage11_foundation_trace_summary.py
  • Integration: apply_foundation_stack.sh --with-tracepoints (optional)
  • What should compile:
    • include/trace/events/kairo.h with 4 TRACE_EVENT definitions: kairo_request_classified, kairo_decode_dispatch, kairo_prefetch_dispatch, kairo_write_demoted
    • #include <trace/events/kairo.h> in block/blk-mq.c and block/mq-deadline.c
    • compile-targeted call site annotations (LINUX-6.8-CHECK)
    • trace call in dd_kairo_dispatch_decode_request()
    • no model/session/backend fields in tracepoint payloads
  • What should be measurable:
    • tracepoint presence in /sys/kernel/tracing/events/kairo/
    • per-request lifecycle trace events (classification, dispatch, demotion)
    • experiment harness detects tracepoints_available=true|false
    • structured results under results/stage11/<timestamp>/
  • What distinguishes Stage 11 from Stage 8:
  • Stage 11: 4 tracepoints, compile-targeted foundation, minimal payloads
  • Stage 8: 9 tracepoints, broad RFC scaffold, model/session/backend fields
  • Stage 11 applies only with --with-tracepoints flag
  • Stage 11 uses LINUX-6.8-CHECK annotations (vs CONCEPTUAL-HOOK in Stage 8)

Stage 12

  • Broad RFC/POC patches involved: 0020
  • Docs: docs/stage12_model_session_fairness.md
  • Scripts:
    • scripts/run_stage12_fairness_experiment.sh
    • scripts/parse_stage12_fairness_summary.py
  • What should compile:
    • struct kairo_fair_entity and struct kairo_fairness_state in mq-deadline.c
    • fairness credit model with per-entity decode credits
    • entity lookup (linear scan of fixed-size array)
    • credit refill mechanism
    • sysfs tunables: kairo_fairness_enable, kairo_model_decode_credit, kairo_session_decode_credit, kairo_fairness_refill_ms, kairo_noisy_session_threshold
    • sysfs counters: kairo_fairness_refills, kairo_fairness_model_throttles, kairo_fairness_session_throttles, kairo_noisy_session_events, kairo_protected_decode_dispatches, kairo_prefetch_fairness_throttles, kairo_write_fairness_demotions
    • conceptual hook points: kairo_fairness_allow_decode, kairo_fairness_throttle_prefetch, kairo_fairness_demote_write, kairo_fairness_account_dispatch, kairo_fairness_refill_if_needed
    • noisy session detection with configurable threshold
  • What should be measurable:
    • fairness counter movement in sysfs when fairness is enabled
    • five canonical experiment cases comparing balanced vs noisy vs fairness
    • structured output under results/stage12/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • benchmark --noisy-session, --noisy-model, --noisy-multiplier flags
  • What is still RFC-only:
    • real entity lookup hash table (linear scan for now)
    • per-device fairness state (uses static/placeholder state)
    • timer-based credit refill (called from dispatch path)
    • tracepoint integration with Stage 8 observability
    • interaction with Stage 10 adaptive controller
    • effectiveness on real multi-tenant AI inference workloads
    • optimal credit pool sizes for different inference patterns

Stage 13

  • Broad RFC/POC patches involved: 0023
  • Docs: docs/stage13_decode_latency_histogram.md
  • Scripts:
    • scripts/run_stage13_latency_histogram_experiment.sh
    • scripts/parse_stage13_latency_histogram_summary.py
  • What should compile:
    • enum kairo_decode_latency_bucket with 10 buckets
    • struct kairo_latency_histogram with buckets, samples, sum_us, max_us
    • helpers: kairo_latency_bucket_for_us(), kairo_latency_histogram_add(), kairo_latency_histogram_estimate_percentile(), kairo_latency_histogram_reset()
    • integration with Stage 10 controller via dd_kairo_controller_update_from_hist()
    • 10 histogram bucket sysfs counters, plus samples and max_us
  • What should be measurable:
    • histogram bucket counts from user-space benchmark samples
    • p95/p99 from bucket estimation compared to true sorted percentile
    • five canonical experiment cases covering various I/O patterns
    • structured output under results/stage13/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • benchmark histogram bucket output fields: decode_lat_0_10us, decode_lat_10_25us, ..., decode_lat_gt_5ms
  • What is still RFC-only:
    • completion-path histogram add call site (CONCEPTUAL-HOOK)
    • timer-based histogram reset for sliding-window estimation
    • bucket boundary tuning for real NVMe devices
    • interaction with Stage 12 fairness
    • tracepoint for histogram snapshot (future segment)
    • merged histogram across devices

Stage 14

  • Broad RFC/POC patches involved: 0024
  • Docs: docs/stage14_controller_feedback_wiring.md
  • Scripts:
    • scripts/run_stage14_controller_feedback_experiment.sh
    • scripts/parse_stage14_controller_feedback_summary.py
  • What should compile:
    • timestamp fields classify_time_ns, dispatch_time_ns, decode_queue_latency_us in struct kairo_request_hints
    • helpers: kairo_mark_classify_time(), kairo_mark_dispatch_time(), kairo_decode_queue_latency_us()
    • call site in dd_kairo_dispatch_decode_request() for decode reads
    • controller feedback counters: kairo_controller_latency_samples, kairo_controller_missing_timestamp, kairo_controller_latency_updates, kairo_controller_histogram_resets, kairo_controller_decode_latency_gt_target
    • histogram-based p95/p99 in dd_kairo_controller_update()
    • histogram reset on control window expiry
    • kairo_controller_sample tracepoint (documented only)
  • What should be measurable:
    • controller feedback counter movement in sysfs on patched kernel
    • decode queue latency fed into histogram on every decode dispatch
    • missing timestamp detection when classify_time is zero
    • five canonical experiment cases
    • structured output under results/stage14/<timestamp>/
    • parser output with counter delta columns
  • What is still RFC-only:
    • timestamp recording in dd_kairo_dispatch_decode_request() (CONCEPTUAL-HOOK)
    • kairo_controller_sample tracepoint (documented, not wired)
    • per-device timestamp tracking (uses per-request metadata)
    • interaction with BPF dispatch hook (patch 0016)
    • interaction with Stage 12 fairness throttle path

Stage 15

  • Broad RFC/POC patches involved: 0025
  • Docs: docs/stage15_fairness_accounting_sysfs.md
  • Scripts:
    • scripts/run_stage15_fairness_accounting_experiment.sh
    • scripts/parse_stage15_fairness_accounting_summary.py
  • What should compile:
    • sysfs show/store functions for 5 tunables: kairo_fairness_enable, kairo_model_decode_credit, kairo_session_decode_credit, kairo_fairness_refill_ms, kairo_noisy_session_threshold
    • sysfs read-only show functions for 7 counters: kairo_fairness_refills, kairo_fairness_model_throttles, kairo_fairness_session_throttles, kairo_noisy_session_events, kairo_protected_decode_dispatches, kairo_prefetch_fairness_throttles, kairo_write_fairness_demotions
    • all counters wired into the Stage 12 fairness hooks as event observations
    • bounds-checked store functions for all tunables
  • What should be measurable:
    • fairness counter movement in sysfs when Stage 12 fairness is enabled
    • tunables accept valid inputs and reject out-of-bounds values
    • five canonical experiment cases covering balanced, noisy, and write-pressure
    • structured output under results/stage15/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • benchmark --noisy-session, --noisy-model, --noisy-multiplier flags
  • What is still RFC-only:
    • dispatch-path integration of fairness hooks (CONCEPTUAL-HOOK)
    • timer-based credit refill (called from dispatch path)
    • per-device fairness state (uses static/placeholder)
    • interaction with Stage 10 adaptive controller
    • interaction with BPF dispatch hook (patch 0016)

Stage 16

  • Broad RFC/POC patches involved: 0026
  • Docs: docs/stage16_blkcg_ai_io_controller.md
  • Scripts:
    • scripts/run_stage16_blkcg_experiment.sh
    • scripts/parse_stage16_blkcg_summary.py
    • kernel/integration/linux-6.8/audit_blkcg_hooks.sh
  • What should compile:
    • enum kairo_blkg_policy_class in include/linux/blk-cgroup.h
    • struct kairo_blkg_policy and struct kairo_blkg_stats in block/blk-kairo-blkcg.c
    • conceptual hooks: kairo_blkg_policy_from_request, kairo_blkg_allow_decode, kairo_blkg_throttle_prefetch, kairo_blkg_demote_write, kairo_blkg_account_dispatch
    • blkcg audit script for Linux 6.8 candidate hook points
    • CONFIG_BLK_CGROUP_KAIRO Makefile guard (scaffold only)
  • What should be measurable:
    • conceptual cgroup policy class model for AI inference containers
    • five canonical experiment cases covering single-tenant, multi-tenant, and background pressure
    • structured output under results/stage16/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • audit script runs against Linux 6.8 source tree
  • What is still RFC-only:
    • real blkcg_policy_register() call (blocked by #if 0)
    • cgroup filesystem interface files (documented only)
    • dispatch-path integration of blkcg hooks (CONCEPTUAL-HOOK)
    • per-blkg_stats allocation (no pd_alloc_fn wired)
    • multi-cgroup fair scheduling
    • CONFIG_BLK_CGROUP_KAIRO Kconfig symbol (does not exist upstream)
    • benchmark --tenant-id / --tenants / --tenant-mode flags (placeholders only)

Stage 17

  • Broad RFC/POC patches involved: 0027
  • Docs: docs/stage17_io_uring_kv_region_hints.md
  • User-space header: include/kairo_hints.h (KV region structs, enums, flags)
  • Scripts:
    • scripts/run_stage17_io_uring_region_experiment.sh
    • scripts/parse_stage17_io_uring_region_summary.py
    • kernel/integration/linux-6.8/audit_io_uring_hooks.sh
  • What should compile:
    • enum kairo_kv_region_type with 6 types in include/linux/blk-mq.h
    • struct kairo_kv_region_hint with region metadata
    • conceptual hooks: kairo_request_has_kv_region, kairo_apply_kv_region_hint
    • IORING_REGISTER_KAIRO_KV_REGION and IORING_REGISTER_KAIRO_KV_REGIONS opcodes
    • io_uring audit script for Linux 6.8 candidate hook points
    • User-space enum kairo_user_kv_region_type and struct kairo_user_kv_region_hint
    • Benchmark --kv-region-id, --kv-region-type, --kv-region-count, --kv-region-size, --registered-buffer-mode options
  • What should be measurable:
    • conceptual KV region hint model for AI runtime memory management
    • five canonical experiment cases covering decode, session, model, recomputable, and many-small regions
    • structured output under results/stage17/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • audit script runs against Linux 6.8 source tree
    • benchmark output fields: kv_region_id, kv_region_type, kv_region_count, kv_region_size, registered_buffer_mode
  • What is still RFC-only:
    • real IORING_REGISTER_KAIRO_KV_REGION opcode handler (no handler wired)
    • kernel region store (no data structure for registered regions)
    • dispatch-path integration of KV region hints (CONCEPTUAL-HOOK)
    • io_uring worker in benchmark (uses pread/pwrite)
    • real registered-buffer tagging
    • region-level override of per-request hints

Stage 18

  • Broad RFC/POC patches involved: 0028
  • Docs: docs/stage18_recompute_aware_eviction.md
  • User-space header: include/kairo_hints.h (eviction policy enum, name helper)
  • Scripts:
    • scripts/run_stage18_recompute_eviction_experiment.sh
    • scripts/parse_stage18_recompute_eviction_summary.py
  • What should compile:
    • enum kairo_eviction_class with 7 classes in include/linux/blk-mq.h
    • struct kairo_eviction_decision with score, reason_flags, metadata
    • conceptual helpers: kairo_eviction_class_from_request, kairo_eviction_score_request, kairo_should_prioritize_eviction, kairo_should_defer_eviction, kairo_account_eviction_decision
    • eviction score policy (10 counters, 8 score constants)
    • User-space enum kairo_eviction_policy_mode and name helper
    • Benchmark --eviction-policy, --eviction-pressure options
  • What should be measurable:
    • conceptual recompute-aware eviction model
    • six canonical experiment cases covering ordinary, recompute-short, recompute-session, model-cache-avoid, decode-pressure-defer, and mixed-eviction-pressure scenarios
    • structured output under results/stage18/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • benchmark output fields: eviction_policy, eviction_recompute_ok, eviction_lifetime, eviction_score_model, eviction_pressure
  • What is still RFC-only:
    • dispatch-path integration of eviction scoring (CONCEPTUAL-HOOK)
    • sysfs counter registration (no kobject for eviction counters)
    • decode-latency feedback to eviction policy
    • per-request access tracking for recency/decode-hot penalties
    • fairness-aware eviction (noisy session vs well-behaved session)
    • real starvation-escape mechanism for eviction

Stage 19

  • Broad RFC/POC patches involved: 0029
  • Docs: docs/stage19_kv_residency_heatmap.md
  • User-space header: include/kairo_hints.h (heatmap mode enum, name helper)
  • Scripts:
    • scripts/run_stage19_kv_heatmap_experiment.sh
    • scripts/parse_stage19_kv_heatmap_summary.py
  • What should compile:
    • enum kairo_kv_heat_class with 6 classes in include/linux/blk-mq.h
    • struct kairo_kv_heatmap_entry with access counters, timestamps, heat score, and classification
    • struct kairo_kv_heatmap with fixed-array region table, decay timing, aggregation counters
    • 8 conceptual helpers: lookup, account_decode/account_prefetch/ account_write/account_eviction, decay, classify, region_evictable
    • heat scoring policy with 4 weights and 3 thresholds
    • 9 sysfs counters (hits, misses, updates, decays, hot/warm/cold/ evictable/protected regions)
    • User-space enum kairo_heatmap_mode and name helper
    • Benchmark --heatmap-mode, --hot-region-ratio, --region-reuse-ratio, --cold-region-ratio options
    • Benchmark output fields: heatmap_mode, hot_region_ratio, region_reuse_ratio, cold_region_ratio, kv_heat_hot, kv_heat_warm, kv_heat_cold, kv_heat_evictable, kv_heat_protected
  • What should be measurable:
    • conceptual KV-cache residency heatmap model
    • six canonical experiment cases covering no-heatmap, hot-decode, cold-recomputable, mixed-hot-cold, multisession, and eviction-pressure-with-heatmap scenarios
    • structured output under results/stage19/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • benchmark models heat class counters based on config ratios
  • What is still RFC-only:
    • dispatch-path heatmap accounting (CONCEPTUAL-HOOK)
    • sysfs counter registration (no kobject for heatmap counters)
    • periodic heat decay execution (no timer wired)
    • fixed-array linear scan (not production-scalable)
    • integration with Stage 17 KV region hints (conceptual only)
    • decode-latency feedback to decay interval
    • fairness-weighted per-session heat

Stage 20

  • Broad RFC/POC patches involved: 0030
  • Docs: docs/stage20_kv_admission_control.md
  • User-space header: include/kairo_hints.h (admission mode enum, name helper)
  • Scripts:
    • scripts/run_stage20_kv_admission_experiment.sh
    • scripts/parse_stage20_kv_admission_summary.py
  • What should compile:
    • enum kairo_admission_decision with 9 decisions in include/linux/blk-mq.h
    • struct kairo_admission_policy with enabled, min_expected_reuse, min_lifetime_ms, max_decode_p99_us, flash_pressure_threshold, flags
    • struct kairo_admission_request with model/session/region metadata, expected_reuse, expected_lifetime, recompute_cost, hot/local/shared flags
    • 3 conceptual helpers: kairo_admission_decide, kairo_admission_accepts, kairo_admission_account
    • policy rules with 7 priority-ordered conditions (RFC heuristic)
    • 10 sysfs counters (requests, accepts, rejects, 4 reject reasons, 3 accept reasons)
    • User-space enum kairo_admission_mode and name helper
    • Benchmark --admission-mode, --expected-reuse, --expected-lifetime-ms, --recompute-cost-us, --flash-pressure options
    • Benchmark output fields: admission_mode, expected_reuse, expected_lifetime_ms, recompute_cost_us, flash_pressure, admission_decision
  • What should be measurable:
    • conceptual KV-cache admission control model
    • six canonical experiment cases covering decode-hot admit, short-lived reject, recompute-cheap reject, model-local admit, pressure reject, shared admit
    • structured output under results/stage20/<timestamp>/
    • parseable summary logs with CSV and pretty-printed tables
    • benchmark models admission decisions based on policy rules
  • What is still RFC-only:
    • dispatch-path admission decisions (CONCEPTUAL-HOOK)
    • sysfs counter registration (no kobject for admission counters)
    • decode-p99 feedback to admission policy
    • per-cgroup admission budgets
    • integration with Stage 13/14 decode latency (conceptual only)
    • runtime flash pressure measurement