Directories mirror the python/freetoken/ subsystem a test primarily exercises —
put a new test next to the module it protects. A test specific to one model
lives in models/<model>/, mirroring python/freetoken/models/<model>/
(models/deepseek_v41/, models/qwen4_exp/). Older model-specific tests still
sit flat in models/ (models/test_glm5_next_model.py), in subsystem
directories (kvcache/test_dsv4_pool.py) or in dsv4/, and will move there.
| directory | subsystem under test |
|---|---|
daemon/ |
freetoken.daemon — torch-free supervisor (import safety, serve manager) |
server/ |
freetoken.server — OpenAI/Anthropic/Responses APIs, streaming, parsers, accounting, maintenance state |
scheduler/ |
freetoken.scheduler — chunked prefill, cache manager, commit/window locking, status reporting, KV usage, cost accounting |
spec/ |
freetoken.spec — speculative decoding: rejection sampling, GDN verify and replay, the verify contract |
kvcache/ |
freetoken.kvcache — paged pools (incl. DSV4's), rebuild, cache-cost accounting; the three prefix caches live in kvcache/radix/ behind a shared reference model (see its README) |
tokenizer/ |
freetoken.tokenizer — tokenize/detokenize request plumbing, thinking-mode resolution |
engine/ |
freetoken.engine — cache budget planning, config resolution, backend gating, graph capture |
kernels/ |
freetoken.kernel / freetoken.layers — attention, rope, fused ops, JIT cache, pinned memory |
attention/ |
freetoken.attention — attention backends against eager references |
layers/ |
freetoken.layers — whole layer modules (mHC, GatedDeltaNet); a layer's kernels and fused ops go in kernels/ |
moe/ |
freetoken.moe — offload cache, CPU/GPU expert kernels, quantized backends (fp8/nvfp4/mxfp4/gguf) |
models/ |
freetoken.models — the registry, and the loading machinery every model shares (sharding, qkv/expert merge, streaming layers into banks); one directory per model for tests specific to it |
mm/ |
freetoken.mm — multimodal processor, placeholder expansion, encoder cache |
checkpoint/ |
freetoken.checkpoint — FTW read/write and replay |
utils/ |
freetoken.utils — HF config loading, ROCm arch detection |
dsv4/ |
DeepSeek-V4 tests that predate models/<model>/ |
e2e/ |
real-server gates: AIME 24/25/26 generation, runtime cache rebuild |
The CLI surface itself is not unit-tested. ft dispatch, ft ctl, ft launch and
install.sh are thin, they change often, and a test written after the fact only
restates whatever the code currently does — running the command is the real gate.
What does live here is the logic reachable through those commands when it has its
own failure mode: server/test_parser_auto_selection.py covers the architecture ->
parser inference behind --tool-call-parser auto.
uv run pytest tests/ # full suite, ~2-4 min on a GPU box
uv run pytest tests/ -m "not slow" # skip the handful of tens-of-seconds tests
uv run pytest tests/kvcache/ # one subsystemGPU-dependent tests skip themselves when CUDA is unavailable. Marlin NVFP4 tests
skip unless vllm is importable (dedicated venv with vllm>=0.14,<0.15).
needs_weights-marked tests skip unless the env var pointing at a real local
checkpoint is set:
| env var | used by |
|---|---|
FREETOKEN_TEST_MODEL |
e2e/test_aime.py — local model directory |
FREETOKEN_AIME_SERIES |
e2e/test_aime.py — aime24 | aime25 | aime26 (default aime25) |
FREETOKEN_AIME{24,25,26}_JSONL |
e2e/test_aime.py — jsonl for the selected series |
FREETOKEN_AIME_REQ |
e2e/test_aime.py — 1-based problem id, comma list (1,3,7), or all (default 1) |
FREETOKEN_AIME_MAX_TOKENS |
e2e/test_aime.py — per-sample token budget (default 16384) |
FREETOKEN_AIME_SAMPLES |
e2e/test_aime.py — pass@N sample count when sampling (default 3) |
FREETOKEN_AIME_MIN_FREE_GIB |
e2e/test_aime.py — required free GPU memory (default 70) |
FREETOKEN_QWEN36_MODEL |
tokenizer/test_mm_tokenize.py — local Qwen3.6 checkpoint (image processor + tokenizer) |
FREETOKEN_QWEN3VL_MODEL |
tokenizer/test_mm_tokenize.py — local Qwen3-VL checkpoint (same expansion, DeepStack family) |
FREETOKEN_GEMMA4_MODEL |
models/test_gemma4_vision.py, tokenizer/test_mm_tokenize.py — local Gemma-4 gemma4 checkpoint (tower parity, boi/eoi expansion; the tokenizer test also takes the 12B gemma4_unified checkpoint) |
FREETOKEN_GEMMA4_UNIFIED_MODEL |
models/test_gemma4_vision.py — local gemma-4-12B gemma4_unified checkpoint (embedder parity) |
FREETOKEN_GLM53_MODEL |
tokenizer/test_mm_tokenize.py — local GLM-5.3-Flash checkpoint (image-token expansion) |
FREETOKEN_DSV4_MODEL |
tokenizer/test_tokenize.py - local DSV4 checkpoint with its native encoding/encoding_dsv4.py; prefix stability and reminder retention, tokenizer only (no GPU or model weights loaded) |
FREETOKEN_INLINE_GLM_MODEL |
tokenizer/test_tokenize.py - local GLM-5.3 tokenizer/template; inline-system preserve policy, prefix stability and reasoning retention (no weights loaded) |
FREETOKEN_INLINE_QWEN_MODEL |
tokenizer/test_tokenize.py - local Qwen3.6 tokenizer/template; inline-system fold policy, prefix stability and reasoning retention (no weights loaded) |
FREETOKEN_MUSE_MODEL |
models/test_muse_glimmer_vision.py, tokenizer/test_mm_tokenize.py, tokenizer/test_tokenize.py - local Muse-Glimmer-30B checkpoint (vision tests; tokenizer-only inline-system regressions) |
FREETOKEN_MINIMAX_M3_MODEL |
tokenizer/test_mm_tokenize.py — local MiniMax-M3 checkpoint (start/end expansion, token budget) |
FREETOKEN_TEST_MOE_CACHE_SIZE |
e2e/test_aime.py — >0 switches to the offload MoE backend with this cache size |
FREETOKEN_TEST_MEM_RATIO |
e2e/test_aime.py — offload-mode memory_ratio (default 0.9) |
FREETOKEN_TEST_SPEC_ALGORITHM |
e2e/test_aime.py — speculative decoding method (mtp, dflash); prints the accept len of each sample (tokens committed per verify step, the bonus token included) |
FREETOKEN_TEST_SPEC_DRAFT_PATH |
e2e/test_aime.py — the draft checkpoint (--speculative-draft-model-path) |
FREETOKEN_TEST_SPEC_NUM_STEPS |
e2e/test_aime.py — draft tokens per step (default 0, the method's own) |
FREETOKEN_TEST_SPEC_MIN_ACCEPT |
e2e/test_aime.py — accept len every sample must reach (default 0, no floor) |
FREETOKEN_TEST_MAX_EXTEND_TOKENS |
e2e/test_aime.py — prefill chunk size (default 8192; 64 chunks the prompt) |
FREETOKEN_REBUILD_TEST_MODEL |
e2e/test_cache_rebuild.py — local checkpoint directory or GGUF file; boots a real server (falls back to FREETOKEN_TEST_MODEL) |
FREETOKEN_REBUILD_EXTRA_ARGS |
e2e/test_cache_rebuild.py — extra server flags, e.g. --moe-strategy hybrid --moe-cache-size 1024 --max-running-requests 1 for offloaded MoE |
FREETOKEN_RIPPLE_MODEL, FREETOKEN_RIPPLE_GGUF |
models/ricursive/test_ricursive.py — local Ripple-65B checkpoint and a GGUF of it (tokenizer, template and config parity; the parsers' round trip through the template needs only the checkpoint) |
FREETOKEN_GEMMA4_GGUF_GLOB |
models/test_gemma4_gguf_rope.py — glob matching a local gemma-4 GGUF file |
FREETOKEN_TEST_MODELS_ROOT |
models/test_quant_config.py — directory whose checkpoints the stored-tensor scheme scan checks (default /mnt/nvme/models) |
FREETOKEN_DSV4_VL_MODEL |
tokenizer/test_mm_tokenize.py - local DeepSeek-V4 vision checkpoint |
FREETOKEN_DSV4_VISION_CKPT |
models/test_dsv4_vision.py - the same checkpoint, for the parity tests against its inference/ reference code |
test_aime.py takes its sampling protocol from the checkpoint's own
generation_config.json (pass@N at the recommended temperature, or a single greedy
sample when the checkpoint recommends greedy). To run an fp8 / offload checkpoint,
point FREETOKEN_TEST_MODEL at the fp8 dir and set FREETOKEN_TEST_MOE_CACHE_SIZE=8192;
FREETOKEN_AIME_REQ=2 FREETOKEN_AIME_MAX_TOKENS=32768 is the offload recipe (req 2 with
a 32k budget).
Budget the token allowance for sampling, not for greedy: on Qwen3.5-35B-A3B-FP8 the three recommended-sampling samples of aime25 req 1 ran 4.1k / 10.7k / 10.0k tokens against greedy's ~4k, and req 2 needs ~21k even greedy — hence the 32k budget for the harder problems.
Run these before opening a PR — they are cheap, and a gate nobody trips rots silently:
FREETOKEN_REBUILD_TEST_MODEL=<small model dir> uv run pytest tests/e2e/test_cache_rebuild.pyA test must be able to fail on a plausible regression of production code: name the future change that would turn it red. If the only answer is editing the test, do not write it. A test earns its place by guarding one of:
- A bug a normal run would not reveal: a wrong number that still looks like a number, a wrong cache slot, tool calls that quietly stop parsing, a crash only one interleaving or one checkpoint hits.
- A property that took reasoning to get right, checked against something independent of the code (see below).
- Bookkeeping that is easy to forget: every registry entry names exports its package has, every registered architecture gets a parser the factories know.
Not admitted: happy-path tests that repeat what the code obviously does,
assertions that restate the branches just written, config tests that echo
fixture values back, stress loops that cannot hit the bug, and tests whose only
check is assert_called* or "no exception".
Weigh that against how the bug would be found otherwise. Application-facing
surfaces — the terminal UI, the access-log filter, the request ring, the status
and health endpoints, the CLI — announce their own breakage the moment anyone
uses them, and they change shape often enough that a unit test mostly restates
today's implementation. Those belong to e2e/ and to actually running the
thing. What earns a unit test is logic that fails silently, like the bugs
listed above. Prefer covering that where it lives rather than through the
surface that happens to expose it.
Writing a test in the same commit as the feature is right; what matters is where its expectations come from. A test that checks the result against something independent — a PyTorch reference, a round-trip, a CPU mirror, a live registry — keeps working when the implementation is rewritten, and can disagree with it. A test whose assertions restate the branch structure the author just wrote can only ever agree, so it never fails on anything except a deliberate change, and then it is edited to match. If you cannot name what the expectation is checked against, the test is unlikely to earn its place. Bug-repro tests should drive the current call pattern of the fixed code, not the pre-fix one; wire-format and kernel tests should assert against an independent reference (round-trip, dequant oracle, CPU mirror), not against the implementation's own output. Registry-style tests should iterate the live registry rather than hand-maintained whitelists, so new entries are covered automatically.
- Extend before you create. Add a case to an existing
parametrizeor table first, then a function in the existing file for that module. In an existing file, reuse its fixtures unless they stand in for FreeToken objects; build new setup with real objects (below). A new file, directory, fixture or helper needs a reason. - One strong case beats several weak ones. Each extra case must guard a different failure. If the test diff dwarfs the change, cut setup and duplicate cases, not the independent check.
- New models. A model's own tests go in
models/<model>/and meet the same bar: parity against its reference implementation, and properties specific to it (chunked prefill against single-shot, its vision tower against the reference tower). What every model needs comes from shared tests: the model adds rows where they keep a table (a new tool-call format inserver/test_streaming_model_matrix.py, a new attention kind inengine/test_attention_backend_matrix.py), and live-registry tests such asserver/test_parser_auto_selection.pyandmodels/test_models_registry.pycover it with no change. Do not copy config, registry or smoke tests into the model's directory. - Remove what a change makes redundant. If a change makes a test wrong, update it. If it makes a test redundant, propose deleting it: a case can go when deleting it lets no regression past the remaining tests.
- Real objects on the lightest real path. Build FreeToken's own objects
(requests, batches, pools) with their real constructors; do not stand them in
with
SimpleNamespaceorMagicMock, set private attributes, or patch methods to reach a state. Objects that need a model to construct (the scheduler, the engine) may be stood in for, limited to the attributes the code under test reads. Fakes are also fine for what comes from outside the engine: an HF config, a checkpoint file, an HTTP peer. A kernel or layer test does not need a server or a model. - Show it catches the bug. Revert only the fix's logic and check that the test fails on the assertion that guards the bug, not on an import or signature error; then restore the fix and check it passes. Put both runs in the PR.
- Deterministic and tight. Unit and kernel tests use greedy sampling or a
fixed seed (the AIME gate samples on purpose, see above). Assert equality
where the result is exact, and keep tolerances as tight as the math allows.
Order a race with events and wait on a condition with a timeout, never a
fixed
sleep. - Never loosen a test to make it pass. No wider
atol/rtol, lower thresholds, retries, or skips to turn a red test green; skipping on missing hardware, packages or weights is fine. Find out whether the failure comes from your change or also happens onmain.