[Doc]: v0.21 beta — compile-cache env var, source-install runtime, and a stale prefix-caching comment are inconsistent with the code

[littlemex]

While doing a hands-on pass over the v0.21.0.1.0.0 beta on a single trn2.3xlarge, I hit three places where the docs (or an in-tree comment) do not match the shipped code. Grouping them since they were all found in the same beta shakedown.

1. NEURON_COMPILED_ARTIFACTS is documented as the compile-cache path knob, but the code never reads it.

It appears in three docs as the way to set/persist the compile cache:

• docs/guides/reference-configuration.md L156 (table: "Path to cache/load compiled models. Skips recompilation when valid artifacts exist.") and the CLI example at L240
• docs/guides/features-guide.md L44, L48, L985
• docs/guides/how-to-profile-workloads.md L228

But NEURON_COMPILED_ARTIFACTS has zero references in the installed package. The compile cache is resolved solely from VLLM_CACHE_ROOT in get_neuron_compile_cache_dir() (envs.py L341; returns $VLLM_CACHE_ROOT/neuron/compile_cache). Setting NEURON_COMPILED_ARTIFACTS has no effect and prints no warning, so a user who follows the docs silently loses cache persistence across container/instance recreation.

2. setup-guide Option A implies pip install -e . gives a runnable environment, but the Neuron runtime (nki) is not installed.

docs/getting-started/setup-guide.md L47: "This installs the vLLM Neuron plugin along with vLLM and all required Neuron SDK packages." In practice Option A installs vLLM (torch 2.11) and the plugin but no Neuron runtime; requirements/core.txt declares none. current_platform.device_name returns neuron (looks fine), but the first model import fails with ModuleNotFoundError: No module named 'nki' — nki is imported unconditionally on the model path, e.g. vllm_neuron/functional/argsort_unstable.py L4. The DLAMI venv (Option B) ships nki and works.

3. A stale TODO in neuron_model_runner.py claims prefix caching is unsupported, but it is supported and already guarded.

neuron_model_runner.py L409-412 says APC is "not yet supported on Neuron / No validation / will fail silently or cause incorrect behavior". But L669 already raises a ValueError requiring segmented prefill for APC, and docs/guides/features-guide.md L166 states prefix caching is enabled by default. The TODO is stale and could lead a reader to disable a working feature.

How this was verified (trn2.3xlarge, v0.21.0.1.0.0, DLAMI venv vllm_0_21_0_1_0_0)

• NEURON_COMPILED_ARTIFACTS: grep -rn across the entire installed site-packages (.py), including neuronxcc and torch_xla, returns zero references; get_neuron_compile_cache_dir() returns $VLLM_CACHE_ROOT/neuron/compile_cache and changing VLLM_CACHE_ROOT does move the NEFF output directory.
• Option A: on a clean Python 3.12 venv, pip install -e . keeps torch at 2.11.0; pip list shows only vllm-neuron among Neuron packages; from vllm_neuron.model.registry import get_models raises ModuleNotFoundError: No module named 'nki'.
• APC: on this release branch the guard at L669 exists and prefix caching produces correct, deterministic output.

Suggest a potential alternative/fix

Small docs+comment PR (no behavior change), sent from littlemex:fix/v0.21-beta-doc-impl-consistency against release-0.21.0.1.0.0:

1. Replace NEURON_COMPILED_ARTIFACTS with VLLM_CACHE_ROOT in the three docs; optionally add a one-line note that the old variable has no effect.
2. Reword setup-guide Option A as a scope statement ("Option A installs vLLM only; the Neuron runtime — e.g. nki — is provided by the DLAMI venv in Option B; recommended for the beta"), version-qualified.
3. Replace the stale TODO with a NOTE that points at the implemented guard.

Before submitting a new issue...

• Searched existing issues.

[github-actions]

Hi @littlemex, Thank you for filing the issue! We will take a look and get back to you.

[aws-sachinda]

All issues have been addressed (PRs merged)