ignis is the Rust inference engine for Qwen models on one NVIDIA RTX card: text models and an image model, one loaded at a time, and a speech model loaded beside whichever it is. It is deliberately specialized, to few model families and to the consumer and workstation GPU generations it is built for, Blackwell (RTX 50, RTX PRO), Ampere (RTX 30) and Ada (RTX 40).
Kernel leaf: The C++/CUDA compute library linked into ignis behind a C ABI. The forward pass of the text and image models and all GPU compute live here; Rust owns everything above the step — scheduling, KV accounting, serving (ADR 0009).
Step ABI: The C ABI between Rust and the leaf, named after its granularity: one call per step (model load, sequence lifecycle, a prefill span, a decode round), not one per operator. Device-resident — no host activation pointer crosses it (ADR 0009).
Program: The leaf-side layer that owns the forward pass and the sequence state, on top of the vendored ops. Ours, not vendored (ADR 0009).
Vendored op: An operator copied verbatim from a named upstream (the reference, or a third party under MIT, BSD or Apache-2.0: ADR 0051) and tracked by a manifest (pinned commit, content hashes, recorded patches). Anything hand-written is not one, and carries no port claim (ADR 0010).
Adapted op:
Our own operator that contains parts copied from a named upstream under MIT,
BSD or Apache-2.0, with its repo, commit, path and licence in the file header
and in kernel/NOTICE. Ours to change; no parity claim (ADR 0051).
Avoid: port, vendored (for an adapted op).
Port claim:
The claim that a kernel file is its named upstream's code, which must be
diffable: only a vendored op, verbatim or under a recorded patch, makes one
(ADR 0010, ADR 0031), and our own ops and adapted ops make none (ADR 0051).
It says where kernel code came from, not what a seam is: codebase-design's
port is the interface at a seam and its Adapter the thing that fills
one.
Avoid: port, adapter (for kernel provenance: say port claim, vendored op,
adapted op); port claim, adapted op (for a seam role: say port, Adapter).
Reference parity: A behaviour ignis computes exactly as the reference does. The default for every vendored op, and not a claim of correctness: the reference can be wrong (ADR 0037). Avoid: faithful, reference behaviour (for a behaviour ignis has changed).
Ignis patched: A behaviour ignis deliberately computes differently from the reference, because the reference is wrong there; each is a recorded patch with a test that tells the two apart. A difference from the reference it causes is a declared departure, not a regression (ADR 0037). Avoid: fork, divergence.
Lane: A concurrent decode slot. Requests hold a lane while decoding.
Lane tag:
The request's own statement of its class, carried as an ignis extension field
on the request and echoed in the request log. Two classes, Interactive and
Agent; absent or unrecognized means Interactive, except on a decision
request, where it means Agent.
Prefill lane: The single global prefill slot: exactly one request at a time holds device-resident prefill progress and consumes chunks, and every other prefill queues behind it. It is a slot, not a promise about decode: whether decoding continues while it runs is decided by prefill/decode interleaving, not by this term (ADR 0018).
Admission state machine: The fairness machinery (protection, backfill class, temporal credit, frontier distance) deciding which request gets which lane. Pages are a dimension of their own: see KV reservation.
Generation cap:
The most a request may generate, reasoning included: its max_tokens
(max_completion_tokens, max_output_tokens), or else the server's
--model-default-max-tokens, clamped to what the prompt leaves of
--model-max-context. Reaching it ends the request with finish_reason: "length".
Cut tool call:
A tool call a length ending stopped before it closed: it reaches the client
with its name and the arguments written so far (ADR 0096).
Avoid: truncated call, partial call, interrupted call.
Deadline:
How long a request may run before the server cuts it (ADR 0058):
server.request_timeout for one answered whole, server.stream_timeout for
one that streams, from its admission (an image job's from its start, never
its queue); 0 is none. A passed deadline cancels the request and frees its
lane.
Partial reply:
What a request past its deadline is answered under server.timeout_reply: partial, the default: what it generated, ended as the generation cap ends
one (finish_reason: "length", a response incomplete with
max_output_tokens). Not a cap: the header ignis-stopped-by and the request
log's cause: timeout say which it was.
Reasoning redirect:
An EOS a request draws while its reasoning block is still open becomes the
block's </think>, and the model goes on to answer (ADR 0048, GitHub #315).
Loop guard:
The server task between a reasoning request's engine event stream and its
consumer that watches the reasoning for a loop (ADR 0052, config
model.loop_guard*, rewind by default).
Rewind:
The loop guard's action: the engine request is cancelled and a
continuation submitted. The client keeps what it received and gets the tag
[LOOP DETECTED, REWINDING n TOKENS / m CHARS · h] in the reasoning, h a
short hash of the struck text; the Playground strikes the m characters before
it through.
Continuation: The request a rewind submits: the same input, its prompt extended by the tokens kept before the cut and a nudge, with what is left of the generation cap and of the thinking budget.
Nudge:
The sentence, in the model's own voice, a continuation ends its prompt with
("Wait, I'm starting to repeat myself. …"): loop_guard_nudge for the first
rewind, loop_guard_nudge_again for every later one. After
loop_guard_retries nudges, the thinking close takes its place and the
reasoning ends.
History repair:
On every incoming request, guard on or off, the m characters before each
rewind tag and the tag itself are removed from assistant reasoning when they
hash to the tag's h, so the model is given back the text it actually saw (ADR
0052). Never applied to any other text.
Holdback:
loop_guard_holdback characters of reasoning held back from the client (0,
off, by default), so a rewind whose cut falls in the unsent part reaches the
client with no tag at all.
KV reservation:
The KV pages the pool has given a request. Its bound is prompt plus
generation cap, never past --model-max-context.
Avoid: allocation (the leaf's word for the mapped pages).
Parked lane: On the growth branch, a decode lane held out of rounds because its room ahead is below one round's largest append and no page can be had yet. It keeps its pages and resumes at the first advance with room.
VRAM budget: The device memory the whole ignis process may hold, weights included: the number Task Manager shows. Either explicit (the operator names it) or derived (the memory free at start minus the VRAM headroom).
VRAM headroom: The device memory a derived VRAM budget leaves to everything else on the card: the desktop and other processes.
VRAM oversubscription: Holding more device memory than the card has free. Windows serves it by paging device allocations to system RAM (Task Manager's "Shared GPU memory") instead of failing, which slows every step; other systems fail the allocation. Avoid: shared memory, spill (a spill is KV-RAM's).
Host-mapped: Kept in pinned host RAM and read by the GPU where it sits, over PCIe, never copied to the device: the 27B's token embedding (ADR 0062). Every read crosses the link, so it suits data a step reads once. Avoid: on the host, offloaded (an offloaded load is a KV pool policy).
Staged: Kept in pinned host RAM and copied to the device for each use, into room the load already reserved, then read there at device speed. Nothing stays on the device between two uses, unlike an expert in the expert cache. Suits data a step reads many times. Avoid: on the host, offloaded.
KV pool policy: How a load sizes its KV pool, chosen by whether every weight is on the device (ADR 0045). A resident load (the 27B, and Flash-Next on a card that holds every expert) gives the pool the rest of the VRAM budget; an offloaded one (Flash-Next with its experts streaming, as on the 5090) reserves the pool first and gives the expert cache the rest.
Trained position envelope: The 262,144 rotary positions the checkpoint was trained over. Not a limit the engine enforces: a sequence past it is served (the attention envelope reaches 1,048,576 keys under hq-e8-2b), it is simply rotated at angles the model never saw.
RoPE scaling:
The text rotary frequency table a load runs on, chosen at load and frozen for
its life. Either none — the linear table, correct through the trained
position envelope — or YaRN at a factor, which rescales that envelope:
the high-frequency pairs keep extrapolating, the low ones interpolate at
1/factor, a ramp blends the band between, and an attention factor m rides
along.
Performance-first: The organizing principle: correctness (a self-check of sane output) is a non-negotiable floor; above it, performance is the #1 objective and tie-breaker for every scope, feature, and kernel decision.
North-star: The holistic objective: the best local engine for coding, for agentic work (an agent and its subagents saturating the GPU in prefill and decode) and as a companion that listens, reads and draws (ADR 0083).
Self-bootstrapping: The dev/test loop, until 2026-11-08, in which the runner used to build ignis (ninfer, running Qwen 3.8-27B) shared the 5090 with the ignis under test; it ended when ninfer left the development machine (ADR 0084).
Reference: The ninfer stack: the source of the artifact, the vendored ops and the oracle recordings, and until ADR 0082 the speed numbers the gates were measured against. A reference, not a target to match token-for-token (ADR 0005 / 0007).
Layer kinds: The model is 64 layers: 16 GQA (full attention) and 48 GDN (linear attention), hidden 5120, intermediate 17408, vocab 248320.
GQA layer: A grouped-query attention layer: 24 query heads / 4 KV heads of 256, over paged KV, rotated by RoPE at the token's position.
GDN layer: A gated-delta-net (linear attention) layer: 16 key heads and 48 value heads of 128, whose recurrence carries the sequence's GDN slot instead of a KV cache.
Fused plane:
A single stored tensor holding several projections stacked by row (e.g.
attention/query_key_gate_value, mlp/gate_up). Its row order is part of the
artifact contract, not a guess.
Flash-Next: Qwen3.8-Flash-Next, the second model: 125B mixture of experts, 6B active, 48 layers in 12 blocks of 3 GDN + 1 QSA, hidden 2560. Served from its own artifact, loaded at start or by a switch, and never beside the 27B.
Expert: One of a layer's 512 routed SwiGLU MLPs (2560 → 640 → 2560); the router picks 10 per token. The shared expert runs for every token, gated by a sigmoid.
Expert projection: An expert's fused gate/up plane or its down plane: the unit the artifact stores contiguously and residency moves.
K: A matrix's bit width in the trellis format, chosen by allocation on calibration distortion: per expert projection on Flash-Next (one of 2, 2.5, 3, 4; mean 2.5 bits per weight), per projection on a 27B artifact whose body format is trellis (2 to 6).
K class: Expert projections of one shape and one K, hence one byte size: eight classes, each with its own slot pool in the expert cache.
Expert pool: Every expert projection, in pinned host RAM for the life of the load. Only an offloaded load has one; a device-only load has none.
Device-only residency: A resident load's expert residency (ADR 0054): the expert cache holds every projection, so there is no expert pool. At load each record is admitted from the artifact straight to its slot, a layer at a time, and no step misses.
Expert cache: The fixed-size VRAM slot pools that hold the projections in use, replaced least-recently-used; a plan line at load. It takes what the KV pool policy leaves.
Pinned hot set:
The calibration-hottest experts an offloaded load keeps in its expert cache
for its whole life (--vram-pinned-expert-bytes: 0 by default, 2 GiB under
the rtx5090 profile; ADR 0055). Each projection is a pinned expert: a
slot past its K class's LRU slots, a hit whenever selected, never evicted or
copied, and no copy in the expert pool, which is smaller by their bytes.
Avoid: "pinned" alone for it: the expert pool is pinned (page-locked) host
memory. Nor "pinned" for the slots a step selects: a step holds them only
until it ends (the kernel's s_held, the policy model's held), held in the
sense a request holds a media embedding — in use until released — where a
pinned expert is never released.
Router lookahead: The next layer's router applied to this layer's MoE input to prefetch its likely experts into the cache.
Demand copy: A step's copy of the projections it selected that the expert cache did not hold (its misses), on the layer's stream: the expert op waits for it. A prefetch copy runs beside the op instead, for the next layer.
Stall: The device time the expert ops waited on their steps' demand copies, per phase (ADR 0017). The wait for a previous step's unfinished prefetch is not in it. Avoid: residency wait (for the stall alone).
QSA layer: Flash-Next's attention layer: GQA 24 query / 2 KV heads of 256 with an indexer that, above 2051 tokens, selects the key blocks the attention reads (sparse); dense below.
Hyper-connection streams: Flash-Next's residual: four streams per token, each sublayer reading a learned mix and writing back through a gated residual.
N-gram table: Flash-Next's hashed n-gram embedding table (320M rows, INT4), a host-streamed object: read row by row from NVMe through a RAM hot-row cache (or held whole in RAM when the hot-row budget covers it), never materialized on the device.
MTP head:
The checkpoint's own one-layer draft head: it combines the target's hidden
state at a position with the embedding of the token after it and scores the
token after that with the target's output head. On Flash-Next (spec
flash-next/07) it reads the trunk's pre-mixer streams and comes in a
companion container. On the 27B (spec #618, ADR 0095) it is the 12 mtp/
objects every 27B artifact carries, W8 on each (451 MB): its stem projects
[the next token's embedding; the target's post-final-norm hidden], one gated
attention layer whose state is one more attention layer of the KV pool, over
the whole context, and a dense SwiGLU. A 27B load built for it keeps that
state and the continuation hidden through every prefill and round. In a
verify round its chain proposes each lane's drafts (the fixed draft), the
first step from the continuation hidden and the pending token, and the round's
commit writes again the head's rows the committed run licenses, so none the
chain built from the head's own hidden stays below the frontier (#621). On an
RTX 30 or 40 card its matrices run on routes of the full-rate rule (#627): the
vendored W8 SIMT kernel up to 8 tokens, ignis's integer MMA above.
Continuation hidden: On a 27B load with the MTP head: the target's post-final-norm hidden at a sequence's last consumed position, one BF16 vector a sequence kept in its pool slot. The head's entry at a position is built from the hidden there and the token after it, so the entry at a sequence's last position waits for the token the sequence consumes next; the continuation hidden is the half of it that waits. Every prefill chunk and every round reads it for its first head row and leaves the next one (ADR 0095). A state section (#622): a restored or claimed sequence carries it and drafts in its first round.
Companion container:
A small .ninfer beside the main one that holds one optional part of the
model, with its own identity, bound only when that part is on: Flash-Next's
MTP head (flash-next-layout.md §13), bound with MTP, and its vision
encoder, bound with vision (#555).
Pair pin:
The companion's record of the one main container it belongs to (model_id,
weights_id, content hash), checked at load; a mismatched pair is refused by
name.
Image model: Qwen-Image-2.1-Turbo, served from its own artifact: a Qwen3-VL text encoder, a 7B diffusion transformer (the DiT) and a VAE. Reached by a switch, never loaded beside a text model.
Image job: One image request from admission to the delivery of its last image.
Target image: The image an image job is generating.
Condition image: An input image an image job is conditioned on (an edit's source or one of its references). Avoid: reference image (Reference is the ninfer stack).
Image session: The Playground's line of work on pictures (spec playground/06): an image pool and its steps, each step a prompt, the image inputs it was sent with and the pictures it made. Kept in the page's image store.
Image pool:
Every picture of an image session — uploaded, made by a step, marked, or
brought from the chat — each named #N for good.
Image input: A picture of the image pool chosen for the next step; the prompt calls the inputs image 1, image 2… in order. One input is one condition image, or two when it is marked as a mask (the picture, then its mask). Avoid: reference, reference image.
Condition prefix: The prompt and the condition images as the DiT sees them before the target image; computed once per image job and reused by every denoising step of every image the job makes. Avoid: prefix, shared prefix (those are a text model's).
K/V run: A span of keys and values stored in 8 bits for the DiT's attention and quantized as one unit: keys smoothed by one mean, values scaled per channel over the whole span. The condition prefix is one run, the target image another.
Condition budget: How many condition-prefix tokens the image model can hold, set at load from the VRAM the rest of the plan leaves. Condition images beyond it are scaled down, never refused.
Denoising step: One evaluation of the DiT over the target image at one noise level, and the sampler update after it. Always qualified. Avoid: step (bare: that is the Step ABI's unit).
Prompt enhancement: An image job's optional rewrite of its prompt by a text model before the image model sees it. Avoid: polish, rewrite.
Revised prompt: The prompt that prompt enhancement produced, returned to the client beside the image.
Enhancer model: The text model that performs prompt enhancement, named by its served id in the config.
Enhancement visit: A switch to the enhancer model and back inside an image job, made when the image model is loaded. It is not a switch for Last switch. Avoid: borrow.
Speech model: A model that takes audio in and gives back its transcript; it never converses. The third model kind, beside the text model and the image model: Qwen3-ASR. Avoid: voice model, ASR model, audio model
Transcription: One server job turning audio into text, from admission to the delivery of its last text: a whole clip, or live from a stream of audio. Avoid: recognition (bare)
Dictation: The Playground act: the user speaks into a composer and the transcript lands at the cursor, editable, never sent by itself. Avoid: voice input
Diarizer: A model that tells the voices in audio apart and gives back who spoke when; it never transcribes. The fourth model kind: Nemotron 3 Diarization, loaded beside the speech model and never resident in VRAM (ADR 0069). Avoid: speaker model, voice model, diarization model
Transcript: The Playground's record of one conversation heard in its Transcribe view: its utterances in order, each with its times, its speaker and its text, and never its audio. Not a dictation: nothing lands in a composer. Avoid: conversation (the chat's), recording, meeting
Utterance: One stretch of a transcript spoken by one speaker, from where speech is heard to the silence, the speaker change or the length limit that ends it. A pause can split one speaker's turn into two. Avoid: turn, segment (a locate's), item (the Realtime protocol's)
Speaker: One voice the diarizer told apart in a transcript, numbered in the order voices first spoke, and renamed by the user; on a call, the microphone's voice is "Me". Avoid: channel (the model's output slot), participant
Notes: What the text model writes about a transcript, kept with it, in sections its kind of conversation asks for, in two forms: running notes and a recap. Not a memory note. Avoid: summary (one of the sections), minutes
Running notes: The notes kept up while a conversation goes on, in sections for talking: what is on the table now, who is who, what was said that adds up. They change a line at a time. Avoid: live notes, minutes
Recap: The notes written once from a whole conversation, in sections for what comes after it: what it was about, the decisions, the action items, the open questions. Avoid: summary (one of its sections), final notes, minutes
Context image: A picture added to a transcript at one of its moments, cropped or blacked out by the user, that tells what the conversation was looking at from then on. Avoid: slide, screenshot, attachment (each is one case of it)
Artifact:
The .ninfer container a model is served from: its weights in the artifact's
body format, its endpoints (token embedding, output head), an optional
grafted DFlash2 module, and frontend objects (tokenizer / chat template).
Several artifacts share one model family.
Body format: The weight format of an artifact's transformer layers: NVFP4, or trellis. Read from the artifact, never configured. Avoid: "3-bit model", "small model" (the family is the same).
Trellis:
The EXL3 weight code (16×16 tiles, the mul1 codebook, Hadamard rotation with
per-channel scales on both sides), bit-compatible with exllamav3's records, at
a K per matrix.
Recipe: The K of every matrix of a trellis body and the bits of its output head. Its bits per weight are the average of those, a property of the recipe and not of the model. Avoid: "2.8 bpw model", quant level.
Object: The unit inside an artifact. The binder must consume every object at bind time; an unconsumed object is a load failure.
Device view: The artifact crate's export for a bound tensor: device pointer, storage layout, format and shape, so the program binds planes directly with no host dequant.
NVFP4:
The weight quantization format of most of the model: E2M1 values, an E4M3
group scale per 16 stored in a separate scale plane
(blockscale-k16-m128x4-v1 layout), and a per-tensor weight divisor.
hq-e8-2b: The reference's HyperQuant KV cache format, and the profile the owner actually runs: a randomized Hadamard rotation, E8-lattice quantization and Rice coding packed into a fixed per-row budget of 64 code bytes plus 8 metadata bytes per (token, KV head). Fixed bytes per token, so page addressing and CUDA-graph address stability are unchanged.
Prompt planes: The BF16 copies of a GQA layer's visible keys and values that the vendored hq-e8-2b prompt route decodes the cache into before a prefill chunk attends: prefill-only scratch, 4,096 bytes per context token up to 262,144 keys (one band). Gone from serving since #335 (ADR 0061): every hq prefill chunk takes the prompt attention instead, and the vendored route survives only as the oracle the route agreement test holds it against.
Prompt attention:
ignis's own hq-e8-2b prefill attention (an adapted op, ADR 0051/0061): two
passes sharing one online softmax, the residual window in BF16 and every
other key through the band in 8 bits (or FP16 with FP16 accumulate, the
fallback; the load option --model-prefill-attention picks one, 8bit or
fp8 naming the 8-bit band). On an 8.x card (RTX 30, RTX 40) the 8-bit band is
the integer band (spec #503, ADR 0080): Q K^T in INT8 and P V on the u8 x s8
MMA, which int8 names on any card and which is the default on an 8.x card; it runs the residual window's prefill pass
too, while decode keeps the window's BF16 rows. Its workspace stops growing
with the context at one band.
Fused prefill:
The 27B trellis body's prefill as it has always run: the trellis decoded
inside the GEMM's inner loop and the multiply in f16 (--model-prefill-linear fp16). The route on sm_120; on an 8.x card a GeForce's decode on the CUDA
cores binds it.
Dense prefill:
The 27B trellis body's other prefill route (spec #503, ADR 0080,
--model-prefill-linear int8): each matrix decoded once per chunk into one
dense INT8 buffer, sized for the largest matrix, and multiplied on the integer
MMA with an INT8 activation. The default route of the 8.x cards (GitHub #515).
GDN prefill: A GDN layer's pass over a prefill span of 64 tokens or more: the chunked delta rule, which carries the sequence's GDN slot from the span's first token to its last. The 27B's has two routes: ignis's own (ADR 0080), every card's route since ADR 0085, and the vendored chunked kernels, which run only when the measurement switch names them and stay the oracle of ignis's; Flash-Next's GDN layers keep their own vendored kernels whatever the switch says. A shorter span, a decode round or a verify round runs the vendored recurrent kernel on either. Avoid: GDN chunked kernel (one route's name, not the pass)
Band: The fixed buffer the prompt attention decodes the codec's keys into, once per chunk and layer: 32,768 keys of INT8 K and E4M3 V^T (64 MiB) with a scale per 32-key tile, walked band after band over a longer history. Not the prompt planes' "band", which was their 262,144-key ceiling.
Residual window: The keys hq-e8-2b attention reads exact instead of decoding: a sequence's first 32 keys (the sinks), a 512-slot ring of recent keys, and the prefill chunk being attended. Per slot, not per page, so it is a state section a clone or a restore carries; a ring bit names no position, so appended columns a sequence does not keep (a verify round's rejected drafts) must have theirs cleared.
KV format: Which of the two KV cache formats a load runs: hq-e8-2b, the serving default from G4, or BF16, retained as the format every correctness oracle runs against. A model-load option, fixed for the life of the load.
Model family:
The architecture an artifact runs as (Qwen3.8-27B, Qwen3.8-Flash-Next,
Qwen-Image-2.1), read from the artifact's own header: a catalog entry may
state it, and the header must agree (ADR 0074). It decides the forward
program, the drafter and the family scope.
Avoid: model (for the architecture)
Model kind: What a model takes in and gives back: a text model takes text and images and gives back tokens, and converses (the 27B, Flash-Next); an image model takes a prompt and gives back pixels (the image model, ADR 0050); a speech model takes audio and gives back its transcript, and never converses (ADR 0067); a diarizer takes audio and gives back who spoke when, and never transcribes (ADR 0069). Above the model family: every family is of one kind. Avoid: modality (the 27B already takes images in), pipeline
Served id:
The name a client puts in a request's model and GET /v1/models reports:
the operator's model.id, or the family's default id when none is named. It
tells artifacts of one family apart; the family never does.
Avoid: model name
Alias:
A served id one catalog entry also answers to on a card that cannot load that
id's own entry, its card class below the entry's or its GPU generation
older: its family's default id, or another entry's served id of the same
family (ADR 0064, ADR 0074, ADR 0079). It resolves only within
one model family — the entry that answers to it holds an artifact whose header
names that family — and only to an entry the card can load. Entries of
different card classes may answer to one id, and where several fit the card
the one of the largest class answers (ADR 0098): qwen3.8-27b is the 16 GB
27B's on a 16 GB card and the 12 GB 27B's on a 12 GB one.
Fine-tune: A catalog entry of a model family whose weights are another party's post-training of the family's checkpoint: the same family, told apart by its served id, and published under its own license, which may not be the family's. The uncensored variant is the fine-tune whose post-training is an abliteration. Avoid: finetuned model (the upstream checkpoint), variant (alone)
Uncensored variant: A catalog entry of a model family whose weights carry an abliteration of the refusal direction: the same family, told apart by its served id, and it does not refuse. Avoid: abliterated model (the upstream checkpoint), graft (the DFlash2 module's)
Catalog: The models the server knows how to fetch: the built-in catalog, shipped inside the binary, plus the operator catalog, a separate file the config names. Avoid: registry (that word is the remote server, as in "a private registry"), model list
Catalog entry:
One published model: its served id, repo, revision, the artifact to load, every
file it needs with its byte count and SHA-256 pinned, the smallest card
class it loads on (card_class, optional: an entry stating none is never
refused by class; every built-in text and image model states it, while the
speech models and the diarizer, which run on the CPU, state none), the oldest
GPU generation it loads on (generation, optional: an entry stating none
loads on every generation the build runs on), the
aliases it may answer to (ADR 0064), its model family when
neither its id nor its aliases name one (family, ADR 0074), and the license
its weights are published under (license: the license's identifier and a link
to its text; every built-in entry states it, an operator entry may; ADR 0078). An
operator entry never reuses a built-in id.
Endpoint: The server a catalog entry's files are fetched from: Hugging Face by default, or a mirror answering the same repo/revision/file paths. Chosen per machine, not per entry. Avoid: registry, hub
Download:
The server fetching a catalog entry's pinned files from the endpoint into its
models folder, resumed where it stopped and verified against the pins. It is
started at start-up, by the model download command, or from the Server
view, one at a time, and never by a switch.
Avoid: pull, install
Removal: Deleting a downloaded catalog entry's files, finished or partial, from the models folder. It never touches the model loaded, loading or being downloaded, nor a file another entry on disk also lists. Avoid: uninstall, delete (for a model)
Switch by id:
An explicit POST /v1/models/switch naming only the served id: resolved as a
known model (switch.known_models, the start model), then a catalog entry on
disk. switch.allow_implicit does not gate it; it gates only the implicit
switch a request triggers by naming a model.
Servable model:
A model a request may name and be served by now, with no download: the active
model, plus every id the implicit switch resolves to an artifact that is a
file (a catalog entry on disk, a known model whose artifact is there). What
GET /v1/models lists, the active one first (active: true), computed at each
request; with switch.allow_implicit off, the active model alone.
Avoid: available model, installed model
Last switch:
How the most recent switch or reload ended (switched, refused, restored,
failed, with the reason), kept on GET /v1/models after the status itself
has moved on. An idle unload and the load after it are not switches and
leave it as it was.
Idle unload:
Tearing the loaded model down after model.idle_unload seconds (default 300, 0
= never) with no request using it and the engine empty, the way a switch tears
one down, while the process and the API stay up (ADR 0049). GET /v1/models
then reports status: unloaded under the same id; the next authenticated
request that needs the model loads it again (status: loading, requests
arriving meanwhile wait) and is served.
Avoid: sleep, suspend, eviction (that word is a sequence's, not a model's)
Instruction message:
A chat message addressed to the model rather than said in the conversation:
role system, or role developer. The model was trained on neither more than
one of them nor on developer at all, so every instruction message the engine
accepts reaches the model as a system block.
System prompt:
The instruction the rendered prompt opens with, where the tools block and the
reasoning instructions attach and a retained prefix is cut: the leading run
of system messages joined into one, or a leading developer message when no
system precedes it.
System message policy:
What the engine does with a system message that is not the first message. By
default a run of them at the start joins the system prompt and a later one
is rendered in place as a block of its own; strict refuses the request instead,
as the model's template would.
Developer message policy:
What the engine does with a developer message: render it in place (the
default), join it into the system prompt, gather every one into a single
block after it, accept only one right after it, or refuse it. Joining and
gathering re-render earlier history when a new one arrives mid-conversation, so
they trade prefix reuse for fewer system blocks; the others never move a
message.
Render refusal: A request whose messages cannot be rendered into a prompt. It is refused before admission, never served as an empty prompt.
Sequence handle: The leaf-owned, opaque per-sequence object created by the step ABI: its KV pages, its GDN slot, its conv taps, its position, its last token, and (P3-03) its penalty-count row. It is what a snapshot captures and a restore rebuilds.
KV page: A fixed-size device page of the paged KV cache, addressed through a per-sequence block table. Page geometry is reported by the runtime and is what the scheduler's KV pool counts.
GDN slot: One sequence's linear-attention state: per GDN layer, 48 value heads × 128 × 128 fp32 (all 48 GDN layers ≈ 144 MiB per sequence), drawn from a slot pool sized by the concurrency.
Conv taps: The per-sequence causal-conv1d history of a GDN layer.
Penalty-count row:
A sequence's per-vocab-entry occurrence count (one int32 per vocab entry),
drawn from a slot-indexed device buffer the same shape as the KV/GDN pools (one
row per slot). Device-side sampling's presence/frequency penalties read and
atomically update it; zeroed at ignis_seq_alloc like every other slot
section.
State section: One part of what a sequence is made of: its KV pages, its GDN slot, its conv taps, its position and last token, its penalty-count row, on a pool built with DFlash2 its drafter window, and on an hq-e8-2b pool its residual window. Each is either shareable read-only (KV pages) or must be cloned per sequence (everything mutable). A pool built for the 27B's MTP head holds the head's attention state in its KV pages (and under hq-e8-2b its layer of the residual window, with its own ring words) and the continuation hidden per slot, a section of its own; every move carries them (#622).
Snapshot blob: The opaque, versioned byte image of a whole sequence that crosses the ABI. Callers ask the leaf for its size and a format version and never see the section layout; a restore validates the header and refuses a stale or foreign one.
Blob identity: What state was produced under, carried in the snapshot blob's header and in the retained entry alike: the artifact's content hash, the KV format, the blob layout version the leaf writes, and the drafter's presence and draft window. It answers whether these bytes may be written into a sequence at all, and a mismatch is refused before one of them moves.
Residency tier: Where sequence state lives, in three levels: Tier 0 — device (VRAM), Tier 1 — KV-RAM, Tier 2 — KV-disk. State moves down a tier when the one above needs the room, and back up to be served. Avoid: host tier (say KV-RAM).
KV-RAM: Tier 1: the host-RAM tier that holds snapshot blobs of evicted live sequences (so they resume instead of re-prefilling) and of retained state (prompt checkpoints, retained prefixes) that left the device. Bounded by a byte budget held whole from the start rather than grown blob by blob — one page-locked arena, blobs placed first-fit inside it, so room for a blob is a free span and not a count of free bytes — two-tier (probation → protected), and written only at a chunk boundary.
KV-disk: Tier 2: the disk tier below KV-RAM (ADR 0045). It takes what KV-RAM gives up, instead of discarding it, and a device victim KV-RAM cannot take.
Live move: A sequence still being served, moved down a residency tier for the room an admission ranked above it needs, and brought back later to resume bit-exact where it stopped (ADR 0045). It is a transfer, not a call: the sequence is in no round from its start until its last window lands, keeping its pages until it has left.
Prompt checkpoint: The whole-sequence state of a request at its generation opener, the point where the rendered prompt hands over to the model, retained after the request ends so a later request whose prompt extends it (the next turn of a chat or of an agent loop) resumes there instead of re-prefilling. Matched by prompt content, never by a session identifier. A request whose schema is described for structured output takes it where its conversation ends, before the description, which the next turn does not share.
Retained prefix: A shared prefix kept alive with no claimant, so a later, non-concurrent request (the next subagent of a burst) claims it as if it were a sibling. It ends at a reuse boundary — for a chat, the end of the system and tools block, floored to whole KV pages — never at the publishing request's whole prompt head.
Reuse boundary: A prompt position where a request's state is kept for a later request to resume from, predicted before prefill because a hybrid can only resume where it captured (ADR 0029). Sources: the end of the system block, a fan-out head, an observed fork, a reuse marker. Avoid: breakpoint, cache point (a prompt checkpoint is something else).
Fan-out head: The reuse boundary at the longest common prefix of a decision fan-out's rendered questions, computed before the first is submitted. Kept only until that fan-out ends.
Observed fork:
A reuse boundary at the end of the longest run of leading content parts
that a recent decision state also began with, keyed by the match key
there. The second request that repeats a run keeps it; the third and later
resume from it.
Reuse marker:
cache_control: {"type": "ephemeral"} on a decision state's content part:
the caller's reuse boundary at the end of that part. It chooses where state
is kept, never what a request may claim — matching stays by content.
Avoid: cache breakpoint.
Lineage: One conversation's chain of prompt checkpoints, and the only name a conversation has. The clients send no session id, so what ties turn N+1's checkpoint to turn N's is the link that does exist: turn N+1 claimed turn N's entry.
Chained prefix: A shared prefix published on top of another. It publishes the pages it warmed itself over that entry, taking over the reference it was holding rather than adding one, so every page is still charged to the pool exactly once and a claimant of the chain shares all of it.
Pages-only link: A chained prefix with no image of mutable state: the whole pages between what a request shares and its generation opener's page floor, handed over by the prompt checkpoint's capture so that something outlives the request to hold them. No request stands at its end, so it is never claimed on its own; it takes no retained slot, and it goes when nothing stands on it any more. Avoid: imageless prefix.
Retained state: Prompt checkpoints and retained prefixes: state no live request needs. It never costs a live request anything — on the device it is always the first thing to go — and in KV-RAM it is discarded before any evicted live sequence.
Waiting claim: What a request still waiting for its admission holds of the state it chose to resume from on the device: its reference on a shared prefix, and the prompt checkpoint its first chunk will copy. A bet not yet cashed, given up like retained state for an entry ranked above the request.
Retained slot: A place reserved at load for one mutable-state image of a shared prefix (retained, chained or still claimed) or of a prompt checkpoint, the same size as a lane's own state (under hq-e8-2b that includes its residual window, which the VRAM plan counts on a line of its own). Two kinds: a device retained slot sits in the device state arenas beside the lanes and is copied device to device; a host retained slot sits in one pinned host block reserved at load and costs a PCIe copy per capture and per claim, but no VRAM. Avoid: retained pool (a ledger of bytes, not a place).
Match key: What a piece of retained state, and every shared prefix, is addressed by: a hash chain over the prompt tokens it covers, with each media item's identity (its content digest and grid) mixed in at that item's first placeholder, so a prefix covering one placeholder has committed to the whole image. Token ids alone are not enough: two same-size images expand to identical placeholder ids.
Prefix reuse: Requests sharing a prefix skip the redundant prefill — concurrent siblings through a shared prefix, later requests through retained state: the shared KV pages are refcounted and shared in place on the device, and the mutable state sections are cloned device-to-device. It never round-trips through host RAM, and sharing pages alone would save nothing, since prefill must traverse every layer to produce the mutable state (ADR 0024).
Shared prefix: The leaf-owned object prefix reuse is built on: the physical KV pages of a prompt head plus a device image of the mutable state at its end, held by a refcount. Every holder's block-table row addresses the same pages, the pool is charged for them once, and they return when the last holder releases, unless it becomes a retained prefix.
Publish point: The chunk boundary a prefix is published at, and 0 for a request that publishes nothing. A shared or retained prefix's is always a whole number of KV pages in — a chained prefix's is measured from the start of the prompt, not from what the publisher already shares; a prompt checkpoint's is the generation opener wherever it falls, and whoever reuses it copies the partial page rather than sharing it. On a load built for the 27B's MTP head the page holding the publish point's last position is every holder's own copy, whatever the point: the head's entry there is built from the token after it, which each holder writes from its own (#622).
Holder copy:
On a load built for the 27B's MTP head, a sequence's own copy of the KV
page holding a shared prefix's or a prompt checkpoint's last
position, mapped in place of the shared page: the head's entry at that position
is written from the token after it, which each holder (the publisher and every
claimant) writes from its own (#622, ADR 0095). The leaf holds at most one copy
a sequence and one a retained slot (a prefix's or a checkpoint's, gone with
its image) past what the ledger charges, so admission keeps that many pages back
from the pool (ignis_core::seq::holder_page_reserve, the plan's
holder_pages, kv_holder_pages in the plan event), and a capture may lend its
prefix's copy beside the pages the ledger moves (the scheduler's
kv_holder_lag: 1 there, 0 on every other pool).
Eviction priority:
The one ordering that decides what loses residency, on the device, in KV-RAM
and on KV-disk (ADR 0023, ADR 0045). A need moves only what ranks below its
requester: retained state first, then Agent before Interactive, then what
is not in the decode round, then the latest-submitted.
Chunked prefill: Prefilling a prompt span through the span+position prefill call in prefill chunks rather than one token at a time. The leaf knows how to loop over a span of any length; it is no longer the only one that does — under interleaving the scheduler hands it one chunk per call and keeps the loop itself.
Prefill chunk: The number of tokens one traversal of the model processes during chunked prefill while a lane is decode-ready: a model-load option, default 1024, a multiple of 128. The prefill scratch is sized for the wider of it and the solo width.
Solo width: The prefill chunk a prompt takes while no lane is decode-ready, a model-load option beside the prefill chunk, never narrower, and equal to it when no wider one is planned. "Solo" names the absence of decoding lanes, not of users: two prompts prefilling at once with no lane decoding both take it. Avoid: idle chunk, burst chunk.
Prefill scratch:
The load's one device region for a step's transient buffers, sized at load for
the larger of what a solo width chunk needs and what a media encode of
one item bound needs (the VRAM plan's workspace line); on Flash-Next for
the chunk alone, the item bound being what it holds with the expert
staging ring (ADR 0087). The two
never run at once, and nothing in it survives from one step to the next.
Avoid: workspace (a user's files), arena (KV-RAM's).
Per-token prefill route: The G1 prefill path that runs the program one token at a time (recurrent GDN, small-T attention). Retained after G2 as a test-only, per-call route: the self-oracle chunked prefill is checked against.
Compute policy:
The per-call activation-precision policy handed to every NVFP4 projection
(A16Only or AllowA4). The engine's policy is the reference's: AllowA4 on
every NVFP4 text projection, prefill and decode alike, with the vendored
per-projection token thresholds deciding the actual route.
Decode round: One step for every decode-ready lane, run as a single batch-wide traversal of the model: the lanes are the batch's rows at one token each, so eight lanes stream the weights once rather than eight times. It is the unit a decode CUDA graph is captured over, per batch width.
Prefill batch: The group of queued requests handed to the backend in one prefill call. A call shape, not a traversal: the runtime walks the group and runs one model traversal per request.
Packed prefill: Several requests' prefill tokens in one traversal of the model (varlen attention, per-sequence GDN state side by side). Not built: the per-chunk synchronization it was to remove measured 0.5% of prefill time (#92). Avoid: prefill batch (for this: naming a group of requests does not make it one forward pass).
Prefill/decode interleaving: Running a decode round between two prefill chunks of the same sequence of work, on the one model stream. Nothing is concurrent on the GPU: the two take turns, so a long prefill costs the decode lanes one chunk of latency instead of the whole span.
Decode share:
The part of the model's time decoding lanes keep while a prompt prefills
(--model-decode-share, in percent). It bounds the lanes' rate during a
prefill, not the gap a single chunk leaves: only the chunk width moves that.
True prefill/decode overlap: Prefill and decode resident on the GPU at the same time, on separate streams, contending for SMs. Distinct from prefill/decode interleaving and deliberately not built: the old roadmap's phase 6, excluded from G3 by ADR 0018 and parked as #545.
Chunk boundary: The position between two prefill chunks: the sequence's KV pages, GDN recurrent slot, conv taps and position have all advanced past the same token, and the chunk's synchronization has returned. A property of the serving loop, and what prefill progress resumes from — resuming is nothing more than scheduling the next chunk.
Snapshot point: A position from which the whole sequence state may be captured to the host tier and later restored. Every chunk boundary is one for the state that exists today; the two terms coincide without being the same property, and a new state section (G4's exact-key side store) must earn the permission again.
Prefill progress:
How far into its prompt a request has been prefilled. It makes Prefilling a
state a request lives in for tens of rounds rather than a moment between two,
so the admission machinery can see a half-prefilled request.
Decode graph: The CUDA graph replayed for a decode round, captured per exact batch width 1..8. Widths are never padded up to a captured one: the GDN slot traffic is per sequence (144 MiB each), so padding a width-1 round to width 8 moves 1.15 GB instead of 144 MB and spends several times the C=1 gate's whole margin.
Per-lane sampling: Sampling parameters and RNG state carried per sequence, not per decode round: lanes in one round hold different temperatures, seeds and penalty histories. Sampling happens device-side in the leaf, which returns token ids: the decode path never ships logits to the host.
N-lane concurrency: The resident decode lanes a load runs at once: 8 (N=8) on a 32 GB card class, fewer on a smaller one, with overflow to the host KV-RAM tier; sized for a ~10-subagent concurrent coding workload. Real at long context only under hq-e8-2b: eight lanes at a 40,960-token context are ~20 GiB of BF16 KV against ~3.02 GB of hq (KV format).
DFlash2:
The 5-layer sliding-window (2048) speculative-decoding drafter (hidden 5120,
draft tokens 1..15, native acceptance 3.4–3.7 tokens/round at 7). Selected at load
(--spec-backend dflash2 --spec-draft-tokens N); without it none of the 66
dflash2/* objects is bound.
Draft window:
The most tokens the drafter proposes per round (--spec-draft-tokens), fixed
for the life of a load: 1..15 under DFlash2 (required; #687), 1..7 under
Flash-Next's MTP head (default 2) and the verify substrate alone; 0..7
under the 27B's MTP head (default 7), where 0 is a load with no speculation,
the load off makes, nothing of the head bound (#623). Past 7 it is the
load's widest window, its extended shape's, and a round at its base
shape drafts 7: the window a round drafts is its round shape's. Not to be
confused with the drafter's own 2048-token sliding drafter window.
Drafter window: A sequence's DFlash2 context: every drafter layer's BF16 K and V for the last 2048 positions (40 MiB). Per-sequence state in the sequence pool, one lane per slot, and one state section of a pool built with the drafter.
Feature taps: The target's hidden states at layers 5, 19, 33, 47 and 61, concatenated and projected into the drafter's context. Taken by a prefill chunk and by the verify traversal; scoped to that chunk or round: never persisted, never a state section.
Drafter floor: The first prompt position whose feature taps a later drafter window reads, which the scheduler hands each prefill job: the earliest point at or past the job's end where a window is read — a prefix it publishes, its prompt checkpoint, its prompt's end — minus the window's 2048. A job taps from the later of its own last 2048 positions and its floor; a job wholly below its floor taps nothing (GitHub #506).
Pending token: The token a sequence has drawn and not yet emitted: the next round's anchor. A prompt's last prefill job draws the first one, and the scheduler emits it at the end of the prefill phase of that step, before any decode round and whether or not a lane is free (#518); the round that follows takes it as its anchor and does not emit it again. Each round after draws the next. Not a draw, which is a constrained decode's emitted token and its probability. Avoid: pending draw, pending successor.
Verify round:
The speculative form of the decode round (P5-04): each lane contributes
k+1 columns — its anchor (the pending token) followed by the drafts, the
tail padded with the anchor — and the 64 layers traverse the batch's columns
once, the GQA layers appending every valid column and the GDN layers recording
instead of advancing. Selected per call through the decode options at the
load's draft window, never padded to it; window 0 is today's round. k is the
window of the round's round shape (#687).
Round shape:
How wide a round's batch is per lane: one column in a decode round,
k+1 in a verify round. A wider shape takes other activation routes
(ADR 0016), so the two shapes can rank two near-equal tokens differently: a
speculative and a plain decode of the same prompt are not byte-identical,
by design. Of a verify round (#686, #687): its window and the most lanes it
runs, the base shape or the extended shape, one for all its lanes,
chosen before the round launches and handed to it; each shape has its own
verify and drafter-append graphs per lane count.
Base shape: The 27B's verify round at 8 columns a lane (7 drafts) at every lane count up to the load's lanes: the one round shape of a load at a draft window of 7 or fewer, and on a DFlash2 load past 7 the shape every round takes unless the shape policy runs it extended from what the shape rule has its lanes ask (#687, #689, #690). Its buffers are the load's: allocated once, each at the larger of the two shapes' sizes.
Extended shape: On a DFlash2 load whose draft window is 8..15, the 27B's verify round at the window + 1 columns a lane, at 1..4 lanes, so that a round never carries more than 64 columns (#687). It runs on the base shape's buffers (a load plans what a 7-draft load plans, plus the wider drafts, at 8 lanes) with its own layout over them, its own host-mapped staging and its own graphs; a kept record of either shape is current only while no later round of either has written the logits they share. Its one-lane attention runs the vendored chunked small-T loop, never the prompt route. Not "wide" or "narrow": those are the Near-tie's computations.
Shape rule:
What a lane of a DFlash2 load past 7 drafts asks of its next verify round,
the base shape or the extended shape, from its own earlier rounds,
which the host has before the next round launches (#689). A lane with no
history asks the base shape; after a base round that drafted the whole 7 it
asks the extended one when at least spec.shape_enter_accepted (7) were
taken, and a base round its budget cut short leaves its ask as it was; after
an extended round it asks while that round took at least
spec.shape_stay_accepted (6). It reads what the target accepted, never what
a stop or a grammar's cut kept, and no confidence from the drafter. The ask is
the sequence's: a snapshot and a restore carry it, a prefix or checkpoint
claim starts without it. The shape policy turns a round's asks into its
shape; a round of 5..8 lanes is never extended.
IGNIS_ROUND_SHAPE=base|extended forces either shape over both, for
measurement.
Shape policy:
How a verify round of a DFlash2 load past 7 drafts takes its round
shape from what the shape rule has its lanes ask (#690;
spec.shape_policy). all: extended only when every lane asks, so no lane
pays for another. aggregate, the default: at one lane the lane's ask; at
2..4 lanes extended when the drafts the asking lanes are expected to take past
position 7 exceed (the extended round's period over the base round's at that
lane count, spec.shape_period_ratios, less one) times the tokens every lane
is expected to take a base round, each lane's expectations read over its own
last spec.shape_estimate_rounds (16) verify rounds; a lane that asks with no
extended round among them is granted one, so the estimate has one to read. A
lane that does not ask then runs the round's 16 columns too and pays for its
neighbour. The lane's recent rounds are host state, not a section: a restore
or a claim starts them afresh, the ask kept. A round of 5..8 lanes is at the
base shape under both.
Near-tie: A split between the token a decode round picked and the one a verify round picked at the same position that the numerics explain. The position is computed outside the rounds in a narrow shape (one token at a time) and a wide one (a chunked span at least as wide as the verify round): it is a near-tie when the two disagree about the winner, when the wide one sides with the verify round, or when the margin is within the computation's own rounding. Only a wide-shape computation that sides with the decode round beyond that margin names a fault of the verify round. Avoid: two probes that both reach the position one token at a time as the judge (neither takes the verify's shape)
Extent (verify round):
The draft columns a lane actually verifies in a round: min(k, proposed, remaining budget − 1, remaining context − 1), where under DFlash2 the drafter
always proposes k. Extent 0 is a fallback step inside the same round, one
committed token.
Committed run: The tokens a verify round emits for a lane: the anchor plus the accepted drafts, cut at the first stop id inclusive (1..k+1). The KV frontier, the GDN slot and the pending token move by exactly the run, so a sequence never stands past the text it emitted. The first round after a prompt commits the anchor its prefill already emitted and emits only the accepted drafts (0..k, #518).
Prefill outcome:
What one prefill chunk cost beyond warming its KV (PrefillOutcome,
GitHub #192): the microseconds the leaf spent encoding a media item,
which is 0 on a text chunk and on one that reuses an embedding an earlier chunk
of the same item encoded, and on the prompt's last job the pending token it
emits (#518). One per job, in order, the way a committed run crosses the
seam per lane; the request log sums the encode into media.encode_seconds.
ReplaySSM record and fold: How the GDN layers speculate without advancing: the verify traversal records each column's conv input, key, value and gates (the records), and after accept the fold replays the committed prefix of each lane's records into its recurrent slot and conv taps in one vendored call. A rejected draft's transition is recorded and never folded — that is the whole rollback.
Drafter:
Whatever fills a verify round's proposal seam: DFlash2 inside the leaf
(P5-05), proposing each lane's drafts from its drafter window in the same
pass that verifies them, or a fake drafter in the substrate's own tests
(verify-only load: the verify substrate with nothing bound, the drafts passed
per call). The 27B has two, one per load: DFlash2, every profile's, and its
MTP head (ADR 0095), chosen with --spec-backend mtp: a load built for it
binds the head and keeps its state current; its chain proposes in the verify
round (#621). ModelFamily::drafters lists a model's, its default first
(#623).
Draft rule:
How a verify round's sampling lanes draft and accept (--spec-draft-rule,
the 27B's, ADR 0090): the fixed draft, the selector walk's one draft per
column taken with the target's probability of it (the vendored accept), or
the sampled draft. Greedy lanes draft and accept the same way under both.
Fixed draft:
The draft rule under which a sampling lane's drafts are the drafter's single
best per column (under DFlash2 the selector walk's), each taken with the
target's probability of it and, on a refusal, replaced by a draw from the
target's distribution without it: the vendored accept's rule, the rule of
speculative sampling for a drafter whose distribution is one-hot. Every round
of the releases before 0.11.0 drafted by it, Flash-Next's MTP rounds draft by
it, and --spec-draft-rule fixed selects it on the 27B.
Sampled draft:
The draft rule under which a sampling lane draws each draft from the
drafter's distribution q over the column's 16 candidates (the selector's
transition scores at the lane's temperature times a factor, cut by a top-p),
and the accept takes it with probability min(1, p/q), drawing from
max(p − q, 0) on a refusal: the rule of speculative sampling, so the text keeps
the target's distribution whatever q is. Its host reference is
ignis_core::sampled_draft, its kernels the leaf's own.
Teacher-forced round:
A verify round whose accept verdict is a recorded text's instead of the
target's draw (GitHub #267; test-only, the decode options' forced_tokens,
which no serving path sets). The pass is any round's — under DFlash2 the
drafter proposes from the lane's drafter window — but the lane accepts the
leading drafts the recording agrees with, at most its extent, and takes the
recorded token after them as its correction, so the committed run is the
recording's. Under the sampled draft (GitHub #577) a sampling lane is forced
too: its drafter drafts the recorded tokens that are among its candidates, and
its accept takes each with the probability a live round under the rule takes
the token it emits, min(p, q)/p — the live round's own verdict where the lane's
draws emit the recorded token, a coin of its own elsewhere — so the rounds a
replay walks are distributed as live rounds under the rule are. On a 27B load
built for the MTP head (GitHub #626) the head's chain proposes in the pass
as in a live round; the head drafts under the fixed draft only, so only greedy
lanes are forced, and fed the text its own rounds committed a forced round is
those rounds bit for bit. On a DFlash2 load of two round shapes (#692) a
forced round may name its shape, which it then runs at whatever would choose
one, and the drafts it verifies in place of its drafter's: offered the recording
itself, every column holds the recorded text and the round commits its whole
shape, the rows the extended round's numerical class is held by (ADR 0104).
Proposal readout: The drafter's proposals at a sequence's frontier, read without running a round (GitHub #267, test-only): the drafts, each draft column's 16 candidates, their proposal-head logits and the selector's transition scores, the lattice the replay walks other selection rules over; under the sampled draft a sampling lane's drawn drafts, with the distribution q each was drawn from (GitHub #577). Named after the readout's precedent — a measurement beside the program, never a second way to generate — but not one: a readout reads named tokens' target logits at one position and costs a prefill; this reads the drafter's lattice and costs one eager drafter forward. Under the 27B's MTP head (GitHub #626) it reads the head's chain instead: the drafts the next round proposes, each with the head's probability of it (its share of the softmax at temperature 1 over the head's scores, the whole vocabulary) and its 16 leading candidates, at one eager chain's cost.
MTP:
The model's native multi-token-prediction head as a drafter
(--spec-backend mtp), its MTP head, chosen at load. Flash-Next drafts with
it (draft window 1..7; a round of several lanes verifies fewer drafts a lane,
down to none, to stay within the load's row budget, --spec-draft-rows).
The 27B drafts with it as its second drafter, in place of DFlash2 (#623),
opt-in on every card class but the 12 GB one, whose profile defaults to it
(ADR 0098): draft window 0..7, the fixed draft only, DFlash2's own settings
refused by name where the operator names them, and drafting after images as
DFlash2 does (#624: its rows rotate at their positions plus the rope delta
and embed an image's column at a placeholder). It saves DFlash2's weights,
drafter window and round, and costs one more attention layer in every KV
page, the head over every prefill position, and a page a holder of a shared
prefix (ADR 0095).
Vision:
Multimodal input: images today, video refused until its own spec. A load
option (--vision-enabled): without it nothing of the tower is bound or
reserved and text serving is unchanged.
Vision residency: Where a vision load keeps the vision encoder's weights and the embedding pool: host (the default: the weights staged for each media encode, the pool host-mapped), so that vision costs no context while no image is in flight; or device (both in VRAM for the life of the load), for a card with VRAM to spare on a host short of RAM.
Media acquisition:
Turning a request's image parts into prepared media items before it is
admitted: decoding a data: URI or fetching a URL under the media policy
(private addresses refused unless --media-allow-private-network, byte cap,
deadline), then preprocessing, memoized by content digest. A refused image
refuses its request before admission; nothing reaches the scheduler. Images
that pass the request's budgets only together are not refused: the oldest
become dropped images.
Media encode: The prefill step that runs the vision encoder over one media item and leaves its media embedding in the embedding pool, in the first chunk that reaches the item — unless the pool already holds that item's, in which case the chunk does no encoding at all. Its wall time is reported on the request, apart from the prefill's.
Media item: One image in a prompt: its patch grid, the run of placeholder tokens it expands to, its BF16 patch rows and the digest of the bytes it came from. The unit the processor prepares, the encoder encodes and a prefill chunk carries at most one of.
Vision tokens:
A media item's merged tokens, one per 2x2 block of patches: the placeholder
run's length, and what the per-request envelope (--vision-max-tokens) is
counted in.
Item bound: The most vision tokens one media item may take: the smaller of the request's envelope and the artifact's pixel bound (16,384 for the 27B and for Flash-Next), so an envelope of 32,768 still bounds each image at 16,384. On Flash-Next it is also no larger than the item whose media encode its prefill scratch and its expert staging ring hold, both idle while an item encodes (#557, #558, ADR 0087); on a card that holds every expert there is no ring, and the scratch alone bounds it. On the RTX 5090 Flash-Next's is the full 16,384: an 8,192-token chunk's scratch holds it alone, a 1,024-token chunk's with the ring (#560). An image over it is downscaled onto a smaller grid, not refused (#248). Avoid: envelope (the request's total).
Image block: A run of a request's images, in prompt order, that together hold at most half of the envelope and half of the media byte budget, or one image past half alone: the unit a request's images are dropped in (ADR 0102).
Dropped image: An image of a request whose images together pass the envelope or the media byte budget, among the oldest image blocks left out to bring the rest within both. The model reads, where it was, a line of text saying an image was there; it is never prepared nor encoded (ADR 0102). Avoid: evicted image, truncated history.
Media embedding:
A media item's encoder output, [hidden, vision tokens], leaf-owned. Held by a request from its encode until the item's last
placeholder is prefilled, and resident in the embedding pool past that,
until something needs its room: a fan-out of N questions over one image
encodes it once (ADR 0035).
Embedding pool:
The load's one reservation for media embeddings, where the vision
residency puts it (--vision-embedding-pool-mib, defaulting to one
envelope-wide item), carved
into fixed-width column pages so an item takes the pages its own columns
need rather than a slot sized for the envelope. An embedding is contiguous in
column space and not in memory: the encoder writes it a page at a time and a
prefill chunk scatters it back a page run at a time, with the same ops either
did over one flat buffer.
Vision encoder: The 27-block tower plus the 2x2 merger that turns patch rows into a media embedding. Run once per item by the media encode step, never per chunk.
Rope delta:
max_position + 1 - prompt_length for a multimodal prompt, whose three-axis
positions advance more slowly than its tokens. Every decode round after such a
prompt rotates at position + rope_delta; the position itself stays the KV
index and the sampler's key.
Decision: One question put to the model about supplied evidence, whose answer is a distribution over a declared set of options rather than generated text. It is the unit a decision request is made of: one evidence, many decisions.
Readout: Reading the logits of named tokens at one position instead of sampling a token there. A readout costs one prefill and generates nothing.
Answer token: The single token that stands for one option of a decision, and whose logit the readout reads. Nothing to do with a decode lane, a GDN slot or a retained slot, which are all per-sequence device resources; an answer token is a vocabulary entry.
Answer alphabet:
The ordered pool answer tokens are drawn from: single characters first
(A-Z, a-z, 0-9), then uppercase bigrams. It is computed from the
loaded tokenizer, never compiled in: a label is admitted only if it encodes to
exactly one token that decodes back to itself, which rejects 114 of the 676
uppercase bigrams in the 27B's tokenizer.
Constrained decode: A decode whose every emitted token lies, per lane, in a declared set of tokens. What a readout is to one position, this is to a run of them: the answer's shape is the caller's, so no parser has to survive a shape the model chose.
Permitted set:
The vocabulary entries one step of a constrained decode may draw from: at
most 32 for a schedule or a forced literal, any number for a
repair. The leaf drives every other column of the logits row to
-1e30 before the vendored sampler runs, so the constraint composes with
temperature, top-k, seeds and penalties instead of replacing them, and what
comes back is a token id like any other.
Schedule:
One permitted set per token a constrained decode may emit, in order. It
is that request's whole generation budget, and its whole stopping condition
unless it names a terminator: the run ends when the schedule is spent, with
stop, and never on EOS, because a token drawn from a set of digits cannot be
one.
Avoid: scheduler (for a schedule: it schedules alphabets, not requests).
Terminator: A token a schedule names, permitted by its steps like any other, that ends the run when it is drawn. Not EOS by another name, and the difference is whose decision it is: EOS is the model saying it has nothing more to say, while a terminator closes the shape the caller asked for.
Draw: One token a constrained decode emitted, and its probability within that step's permitted set — a restricted softmax over at most 32 logits, computed on the device and temperature-free on purpose: it is the model's confidence in the token, not the chance the sampler happened to pick it. A run's draws, in order, are its trace.
Forced literal:
Tokens a request is made to generate, one permitted set of one per round,
after which it is released and generates freely: not a schedule, which
ends its run when spent. tool_choice: "required" or a named function forces
the opening of a tool call this way (GitHub #286), from the first token with
thinking off and right after the reasoning block with it on; the thinking
budget's forced close is the same shape.
Grammar: The shape a caller gives a whole answer, or a held tool's call arguments, as rules that say which continuations of what has been written so far are valid. The third kind of constrained decode, beside a schedule and a forced literal, and the only one that depends on what the model wrote: it checks each sampled token instead of restricting the sampling.
Miss: A sampled token the grammar rejects, caught before the model consumes it.
Repair: Sampling a miss's position again from the same scores, restricted to the permitted set the grammar allows there: the only step of a grammar-constrained decode that sends a permitted set down. Avoid: retry, resample
Structured output: An answer a grammar holds to valid JSON, or to a JSON schema the caller sent. OpenAI's name, kept because the field is OpenAI's. Avoid: JSON mode, guided decoding
Held tool: A function tool whose calls' arguments a grammar holds to its parameters, in whatever order the model writes them: a strict tool, or a tool whose schema the server normalized into strict form. Avoid: constrained tool call, guided tool call
Strict tool:
A held tool a request sends with strict: true, whose schema must lie in
OpenAI's strict subset or the request is refused. OpenAI's name, kept because
the field is OpenAI's.
Avoid: strict function calling
Normalized tool:
A held tool sent without strict whose schema the server rewrote into
strict form, its objects closed and its optional parameters left out of the
arguments when the call leaves them out: OpenAI's default on the Responses
API, and every tool not sent strict: true under the operator's held. Its
length bounds are left out, outside the strict subset (GitHub #641). A
schema that cannot be normalized leaves its tool free, never refused.
Confirmed close: The rule a held tool's call is read by: a string value ends only at a line that is a valid next step of the call, the opener of a declared parameter not yet written, or the function's and the call's closers once every required parameter is written; any other line that looks like a tag is the value's text. A tool that is not held keeps the first-occurrence cuts: each value, body and call ends at the first closing tag written. Avoid: smart close, schema-aware parsing
Answer mass:
How much of the whole next-token distribution the answer tokens hold,
exp(logsumexp(answer logits) - logsumexp(all logits)). It is what says
whether a restricted softmax is reading the model or renormalizing noise: a
decision whose answer mass is near zero has an argmax that means nothing.
Attention readout:
Reading attention heads' pre-softmax scores (q · k / sqrt(head_dim)) at a
prefill's last position over one span of its keys — an image's placeholders, or
a text state's tokens — instead of the logits of answer tokens
(GitHub #260, #263, #264 and #275; ADR 0038, ADR 0039, ADR 0040 and ADR 0041).
A point, a head box or a locate answered this way is a decision:
one prefill, no decode round, no lane.
Pointing head:
The attention head that names which object a point or a head box is
about: L39.h10 (GQA ordinal 9, query head 10) on the served 27B, read at the
position after the forced {"x":. A calibrated constant keyed to the
artifact's content hash: chosen on labelled scenes by cross-validation, never
computed at load, and never read on an artifact nobody calibrated it for — such
a load answers point with the digit chain.
Head set: The attention heads read together beside the pointing head, one image cell each: where that head looks hardest, the fallback cells skipped (GitHub #263, ADR 0039). Each reads a different part of the object the question asked for — its boundary, mostly — so the cells near the pointing head's point outline the object.
Fallback cells:
The image cells attention heads peak on when nothing in the image matches the
question: the first and the last image cell on every grid, and on a grid the
calibration measured, the cells it found there (on 32x32 also (0,1) and
(6,31)). The head set's argmax skips them; the pointing head's map
keeps them, and a map flat enough to fall back to one (region.share about
0.03) is nothing found.
Sub-cell peak: Where a head set's head looks hardest inside its image cell, not just which cell (GitHub #264, ADR 0040): parabolic interpolation over its score at its peak and the two in-line neighbours on each axis, in cells, zero when a neighbour is off the grid or the triple is not a strict peak. It is what lets a box frame an object smaller than the cell grid can express — a 1024 px button is 1.44 cells tall, and a box on cell geometry cannot clear the digit chain at that size.
Allowance:
What the extent grows the kept heads' span by on each side: 0.05 + 4.5 / n cells, n the distinct cells they peak on (never under three, the fewest
measured). The span is a lower bound on the object — it reaches only the
parts the heads hit — and what one part leaves unobserved falls as more parts
are seen: on synthetic scenes the set tiles the object (17-56 distinct cells),
on a photograph it piles onto one distinctive part (6-13).
Locate:
The /v1/decide primitive that names the segment of a text state an
instruction asks for (GitHub #275, #278; specs 18 and 22; ADR 0041, ADR 0042),
with nothing written into the state and nothing generated. Two methods: the
shortlist (the default) and the head vote.
Shortlist:
A locate's default method (GitHub #278, ADR 0042): the calibrated heads
narrow the text to a few candidates — 5 templates at a fold's level 1, 16
rows at its level 2, 16 lines, sentences or records otherwise — and a
labelled choice over them decides, with no token generated. The heads narrow
a very long text; the choice resolves near-duplicates, which it does well
among a handful and badly among hundreds.
End reading: A shortlist's reading of logs and records: a segment scored by where it closes — its last key, its separator and the next segment's first key — each head's lift over the content-free baseline standardized over the window's segments and summed over the end heads. At length a line is marked at its end, not its start.
Sum reading: A shortlist's reading of prose: a segment scored by all its keys, as the head vote does, standardized and summed over the sum heads rather than voted.
End heads:
One of the two head sets of a locate calibration (spec 22): 32 heads of GQA
layers 35 to 55 with the best single-head top-1 at a line's closing keys on
development sets (spec 23). Keyed to the artifact's content hash with
LOCATE_WINDOW_KEYS, as the sum heads are.
Sum heads:
The other head set of a locate calibration (spec 22): the head vote's 32
heads.
Template fold:
A locate's template_fold compression (spec 22): a log's lines grouped
Drain-style into templates — same source label, same token count, tokens
agreeing on half the positions, times removed and obvious variables masked — so
a long log is read as a short text first. Reversible: every level-2 row
maps back to the lines it stands for.
Level 1:
A template fold's first text: one line per template, its variable slots
showing their distinct values while they fit 600 characters, and (xN).
Level 2:
One template's lines as their values only, values first and the line's time
last, exact repeats one row with (xN). The shortlist ranks rows by their
values and shows the last choice each row's first original line, because
whether a line answers cannot be read from its values alone.
Window:
A stretch of at most LOCATE_WINDOW_KEYS (200,000 on the served 27B) keys of a
text the heads read, cut at segment boundaries — at an empty segment where one
is in reach (a paragraph break), else before the segment that would not fit —
and read as its own prefill with its own content-free baseline; windows'
scores are standardized and merged. A text's length is then bounded by the
request, not the context.
Pointer:
Every candidate of a locate's last choice at a share of 0.05 or more, the
pick always first: one answer, or the several segments a two-part answer needs.
Records array:
A JSON array of two or more elements, every one an object, and not the
content-parts shape (every element with a string type): what auto reads as
records, without a statistic.
Found:
Whether a locate's text answers at all (spec 22 § Not found): the last
choice asked again with a "none" option, and for a folded log a yes/no; 1 - p(none), averaged with p(yes) for a log. Below 0.5 the answer names no
segment and keeps its ranking.
Segment:
One place a locate can answer with: a line of a string state, split on \n
exactly (a \r stays part of its line), or an element of an array, within an
optional JSON Pointer. Numbered the way the caller would number them — the
0-based index of split("\n"), the array index.
Head vote: How a locate is read (spec 19's track L, GitHub #276): each of 32 heads of GQA layers 35 to 63 names the segment whose share of its softmax over the span the question raised most above the content-free baseline's, and the most-voted segment wins, a tie to the segment the best-ranked head named. A calibrated constant keyed to the artifact's content hash, like the pointing head: the heads with the most training hits on development sets, never computed at load.
Content-free baseline:
The same prompt with the instruction replaced by N/A (ICR's, QRHead's and
contextual calibration's "N/A"): what the heads do with the state and the
scaffold when nothing is asked — sinks, first lines, position — which the
head vote subtracts before a head votes. One per locate target in a
request, shared by every locate over it; it claims the state's prefix like
any sibling, and costs input tokens but is no decision of its own.
Extent (head box):
The box the kept heads of a head set span: the heads within three times the
median distance of the pointing head's point, at their sub-cell peaks,
their 15%-85% quantiles on each axis grown by the allowance, clamped to the
image. A head point is its centre and carries it as extent; a head box is
it.
Stream:
One of the independent request sequences a Responses WebSocket multiplexes
(OpenAI's stream_id; the connection's default stream when it names
none). Its requests run in order and never overlap; different streams run side
by side.
Avoid: lane (for a stream: a lane is a decode slot, and the stream is the
standard's word, nothing in the engine).
Continuation cache:
What one Responses WebSocket remembers so a request can name
previous_response_id and send only new items: per stream, its latest
finished response and the whole conversation that response saw. It chooses
nothing about retained state: a continuation re-renders that conversation, and
reuse stays a content match on the prompt checkpoint and retained
prefix (ADR 0029).
Warm-up request:
A request that ends where its prefill ends and generates nothing, like a
decision, but reads nothing out and keeps what an ordinary request of the
same prompt keeps: the retained prefix at its system block and the prompt
checkpoint at its generation opener, for the turn that follows. The Responses
WebSocket's generate: false.
Avoid: prewarm (for the request kind; it is the client's word for the act).
Socket admission queue:
The one server-wide first-in, first-out queue a Responses WebSocket request
waits in when the engine is full, told so with response.queued, instead of
being refused with a 503 as an HTTP request is. Only its head retries, each
time a request leaves the scheduler.
Idle socket:
A Responses WebSocket with nothing running, queued or waiting on it and no text
or binary frame from its client since its last response ended; a ping does not
make it busy. One idle for server.socket_idle_close is closed by the server
(GitHub #329), since an open socket holds a Knative pod.
Harness: The program a coding agent runs in (opencode, qwen-code, pi, DeepSeek Harness): it calls ignis as its model and may call its tools. Avoid: client (any program calling the API), agent (the model acting inside the harness).
Connector: What ignis offers a harness beyond chat: its MCP server's server tools, and the skill whose scripts carry the files. Avoid: plugin, extension (a harness's own native APIs).
Server tool: A tool ignis itself offers a harness and runs when it is called; not a tool a client declares in a request (a held tool, a strict tool). Avoid: MCP tool (the protocol's word), ignis tool.
Field:
One configurable setting, declared once (config::schema): its flag
--<group>-<field>, env var IGNIS_<GROUP>_<FIELD>, config-file key
<group>.<field>, default, rule and help text all come from that one
declaration. Fields are grouped by concern (server, model, vram, reuse,
switch, spec, vision, media, download, ngram, kv_disk).
Family scope:
A field both model families take but want sized differently has a variant per
family (qwen38, qwen38flashnext, from ModelFamily, never a served id):
--qwen38flashnext-reuse-kv-host-pool-bytes. A tiebreaker within each
source, not a source of its own: the command line wins whatever its scope.
Applicability:
The families a field means anything for (Only(Flash-Next) for the expert
cache's floor, Only(27B) for vision). An explicit value for a family that
cannot take it is refused at start and dropped, with a log line, on a switch;
a value equal to the field's default is neither.
GPU generation: The family of NVIDIA cards whose features the kernels and routes are reasoned about by: Ampere (RTX 30, compute capability 8.6), Ada (RTX 40, 8.9) and Blackwell (RTX 50 and RTX PRO, 12.0). The word a user meets. Avoid: architecture (the glossary's word for a model family), arch.
Compute capability:
The version a card reports to CUDA (8.6, 12.0), read at start. It says
whose machine code runs on the card and which routes the server takes there.
CUDA architecture:
One target the kernel leaf is compiled for (86, 120a), as
IGNIS_CUDA_ARCHITECTURES names it. Plain code runs on every compute
capability of its major from its own up; an arch-specific a target runs on
its own alone. A start on a card none of the build's entries runs on is
refused by name.
Avoid: arch, SM version.
Card class: The device memory a profile is shaped for, by NVML's total: 12 GB, below 14 GiB (RTX 5070, RTX 4070 family 12 GB, RTX 3060 12 GB, RTX 3080 Ti; no lower bound, a smaller card is read as the class and refused by its plan), 16 GB, 14 up to 20 GiB (RTX 5060 Ti / 5070 Ti / 5080, RTX 4060 Ti 16 GB / 4080), 24 GB (RTX 3090, RTX 4090), 32 GB (RTX 5090), 96 GB (RTX PRO 6000) (ADR 0063, ADR 0098). Memory alone: the GPU generation is a separate axis that decides routes, never the class. It decides which artifact, how many lanes, which drafter and what context the profile defaults to. Avoid: GPU tier.
Profile:
A named bundle of hardware-shaped field defaults (--profile, built-in
rtx-12gb, rtx-16gb (also accepted as rtx50-16gb), rtx-24gb, rtx5090
or rtxpro6000, or a config file's profiles:):
below the config file, above the hardcoded defaults — a default, never an
override. auto, the default, picks the built-in profile of the card's card
class from NVML's total memory (ADR 0063); a named profile always wins. A
profile may set the ladder's step too: rtx-12gb steps 16,384 (ADR 0098).
Adaptation:
A 27B load lowering the values its profile or the hardcoded defaults gave, in a
fixed order, until its VRAM plan fits (ADR 0063): the drafter's shortlist
head to full, then the context down its ladder, then one decode lane (a
load at one lane already, as under rtx-12gb, has no lane step). Only a plan
refused below its minimum adapts; one that fits as configured changes nothing.
Ladder:
The contexts an adaptation plans, from the configured one down every multiple
of the context step (model.adapt_context_step, 65,536; 16,384 under
rtx-12gb) to one step, keeping
the largest whose KV pool holds one full sequence and a spare reservation
(vram.adapt_spare_tokens, 65,536) at more than one lane.
Avoid: step (bare) for one of its rungs: an adaptation taken is an
adaptation step.
Live change:
A PATCH /v1/config that touches only server-level values (timeouts, message
policies, thinking defaults, switch knobs) and no model.
Reload:
A PATCH /v1/config that runs the whole patch through the model switch,
reloading the same model with the patched configuration. One patch is a live
change or a reload, never both.
Canonical event: The one structured representation of a log occurrence (severity, event name, body, attributes, trace context). Formatters are presentation, never the source of truth (ADR 0011).
Event name:
A stable, class-level identifier (ignis.<subsystem>.<event>, e.g.
ignis.model.loaded), never occurrence-specific data.
request.id:
Reused as the OTel trace_id for the request's span tree (root span at HTTP
ingress, children per admission/prefill/decode round/MTP
verify/completion); one identifier for the same causal unit across fairness,
logs, and spans (ADR 0012).
Hot-path logging constraint: Normal INFO operation must not emit one record per token/layer/kernel/allocation on the prefill/decode path; any logging or tracing change touching that path must pass the G4 performance gate (ADR 0007) before merge, not just design review.
Hotpath lint:
The grep-based, hand-maintained scan over a fixed list of
prefill/decode/CUDA-graph-adjacent files that fails cargo test on any
INFO-or-above tracing call or raw println!/eprintln!/print! found
there.
Monitor:
The Playground's live dashboard over the Prometheus exposition (GitHub #165):
the browser
scrapes /ui/metrics, keeps a rolling in-memory history, and derives rates,
latency quantiles and a plain-words verdict from ADR 0017's contract metrics
only. Offered only when metrics are on; not a replacement for Prometheus, and
gone on reload.
Playground: The opt-in browser page ignis serves for working with its models by hand — chat, pictures, speech and workspaces — with per-request figures the browser measures itself, and the Monitor. The user's workbench over the engine: not an operator surface (that is Prometheus), and never a source of facts the engine does not already expose.
Workspace:
A named set of workspace files the user and the model work on together,
kept by the page and never by ignis. Born with its first file, it outlives the
chat that made it: a chat works in at most one at a time, its own or one the
user picked.
Avoid: project, library, artifact (the .ninfer container)
Workspace file: One named item of a workspace — text, a picture or audio — that the model and the user both read and change; "file" alone when the workspace is clear. Avoid: artifact, document
Memory note: A note the Playground keeps across every chat, a title and a body: the model sees the titles and reads a body when one looks relevant. Never part of a workspace. Avoid: memory (bare: that is the device's), workspace file
Turn notice: The instruction message the page adds after the history at a turn: at a chat's first turn the titles of the memory notes and the files of its workspace, later only what someone else changed since, and the moment when it is live. Kept with its turn once sent, so the system prompt never changes. Avoid: context note
Gate:
A milestone acceptance measured on the GPU, never on CPU tests. G1
correctness floor, G2 prefill, G3 decode + serving loop, G4 the reference
feature floor, G5 speculative decoding (docs/archive/ROADMAP.md). The
phases ended with G5 (2026-09-14); an artifact or a route is now admitted by
ADR 0082's gate: the KLD against BF16, DFlash2 on against off and the loop
rate.
GPU profile: The explicit test profile for GPU work: it requires the 5090 free and fails, never skips, when the GPU is busy or a kernel errors.
GPU lock: The one claim on a card, naming the run that holds it, its worktree and its command: whatever goes onto the card (a GPU profile, a server, a GPU command run by hand) holds it for as long as it runs, and it is free once that run has ended. It stands beside the profile's preflight pass, not in its place. Avoid: card lock, GPU mutex
Generated token:
A token the engine decoded, on either channel: the answer
(delta.content) or the thinking trace (delta.reasoning_content, on by
default). Every throughput and latency cell counts generated tokens, because
both channels are the same decode work; a correctness check reads the answer
channel alone, because scratch work is not an answer.
Canary suite: The fixed set of short, high-signal prompts used to detect divergences. Sanity is judged on everything the turn generated: a turn whose budget ran out mid-thought produced sane tokens, not empty output.
Canary oracle: The recorded fixture the canary suite is checked against: the reference engine's greedy (exact-argmax) completions on those prompts for the same artifact, tokenized with the artifact's tokenizer. It admits no artifact since ADR 0082: the KLD against BF16 does.
Teacher-forced agreement: How the canary oracle is scored, and the G1 correctness floor: at each of the first 32 positions the engine is fed the oracle's own token prefix, and its greedy next-token argmax is compared with the oracle's recorded token. The floor is ≥ 95% overall (ADR 0014).
Free-running agreement: The older scoring, where the engine generates its own continuation and the two token streams are diffed. Diagnostic only (ADR 0014): after the first divergence the engines no longer share a prefix, so it measures continuation similarity, not forward-pass health.
Regression set: A fixed set of cases, written or contributed, run through a public endpoint in the GPU profile, each with a tier: a gate case is asserted, a watch case and a known failure (the right label, which the model answers wrong) are printed. A case is a gate only when it sits clear of the decision line on every route the card runs, not on one: a case near the line falls on either side under any change of the numerics, and asserting it reports the change, not a fault. The KLD gate judges the numerics; a regression set catches a feature that broke. Avoid: entailment gate (the set is not a gate)
TTFT cell: One fixed prompt length (8K, 32K) at which time to first token is measured through the bench, the same way against either engine: streaming, greedy, thinking off, a small output budget, deterministic synthetic prompts of exactly that many tokens. Every sample is a cold prefix: each sample (the warmup included) gets its own prompt, distinct from the first content token, so no prefix cache or host KV tier on either engine can serve it; the engine's own computed-prefill-token count must equal the prompt length or the sample is void.
ITL measurement window: The stretch of a G3 ITL cell in which every decode lane was alive: it opens at the last lane's first token and closes at the first lane's end, and only the prefill windows closing inside it are pooled against. A lane the harness cancelled ends at the cancellation, since it was generating until then; a lane the engine ended — its own EOS, the safety cap — ends at its last token.
Live/live gate: How G2 is judged: ignis and the reference measured on the same TTFT cells in the same session on the same machine, and the ratio ignis/reference must be ≤ 1.5. A committed reference record exists for regression and sanity only; it never decides the gate. Historical: ADR 0082 superseded the gates against the reference.
Measurement session:
The identifier both engines' TTFT records carry when they were measured back
to back in one sitting. It is what makes live/live checkable rather than
asserted: ignis-bench g2 refuses a verdict when the two records do not share
one, which is also what stops a committed fixture from ever being the live
side.
TTFT record:
One engine's measured cells plus the identity a later reader needs to audit
them: every sample and its computed-prefill count, the median, the endpoint,
the engine, the artifact, the profile, the date and the measurement session
(ignis-bench ttft --out).
f64 layer reference: A CPU fp64 computation of one GQA layer and one GDN layer on the real weights, the tolerance target for the leaf's per-layer output at G1.
Performance gate (99%):
The acceptance criterion at G4: ≥ 99% of the reference's performance
(throughput / latency) on the trace-replay load, not token-agreement;
correctness is self-checked (sane output, same model, greedy, fixed seed). It
is the first of a ladder of performance gates: G5 asks ≥ 99% of the
reference's committed tok/s under DFlash2, and the phase beyond the reference
gates each item on its own (docs/archive/ROADMAP.md). Historical: ADR 0082
superseded the gates against the reference.
Trace replay: Re-sending a recorded "1 main agent + N subagents" load trace against both engines to compare scheduler behavior, scored per lane tag. The trace is the load, not the baseline: the reference's own run is measured live in the same session (live/live gate), over at least two launches per engine (ADR 0021), and the verdict is the pooled cell statistic.
Load trace: The recorded agent session a trace replay sends. It holds real working content, so it is never committed: its SHA-256 and its shape (request count, class split, arrival span, prompt-length distribution) ship with the gate artifacts, and both run records carry the hash so a reader can prove the two engines replayed the same load.