The adapter selects an installed profile by its opaque owner_mode name and
sets owner_mode_scale in (0, 1]. The inference engine applies that selection
only after authenticating the request. Profile files are installed separately
and scoped to a model revision and serving format.
This is the raw engine contract. An Always On FlashAlign gateway selects and signs every routed request, including all tools and subagents, before it reaches this boundary. A caller that sends requests directly to the raw engine is outside that gateway's policy. See direct access.
The setup wizard can read GET <model-base-url>/owner-mode/profiles. An engine
that implements this optional extension returns:
{
"data": [
{
"name": "example-profile",
"model": {"id": "publisher/model", "revision": "pinned-revision", "format": "FP8"},
"loaded": true
}
]
}Only loaded profiles are offered. Explicit model.aliases can identify the
same served model under additional IDs. The /models response may include
revision and format fields so the wizard can reject mismatches, and input,
contextWindow and maxTokens to configure the model descriptor.
This is a discovery extension, not an upload API or an existing-engine support claim. A 404/405 falls back to local profiles or an operator-entered profile name. The wizard independently checks standard and selected modes with two one-token Chat Completions calls, including the required signed owner request. It never assumes a listed or structurally valid profile is active.
The engine and adapter share a protected 32-byte signing key. The header is:
X-Owner-Mode-Auth: v1:<unix_seconds>:<random_nonce>:<hex_hmac_sha256>
Sign timestamp + "\n" + nonce + "\n" + sha256(raw_body) with HMAC-SHA256.
The engine enforces a 60-second timestamp window and rejects nonce replay
throughout that window. Replay-store failure must fail closed. Keep keys and
authentication headers out of logs.
The bundled Python engine integrations persist keyed nonce hashes and expiry
times across API restarts. vLLM and SGLang use
$XDG_STATE_HOME/flashalign/replay/nonces.sqlite3, or
~/.local/state/flashalign/replay/nonces.sqlite3 when that variable is unset.
FLASHALIGN_REPLAY_STORE can select a stable private path. The llama.cpp
launcher keeps the store in its managed state directory. Keep this state while
reusing the deployment key. Its directory must be owned by the serving user
with mode 0700, and its database must have mode 0600.
TensorFold uses a private replay directory selected with
--owner-replay-dir; direct launches default to a sidecar beside the key file.
The wizard chooses a stable per-key directory outside versioned runtime
folders. The store holds an exclusive writer lock and durably saves accepted
nonce digests before authorizing a request. Preserve it across restarts and
upgrades. A new signing key is required when deliberately resetting this state.
The stores contain nonce hashes, expiration times and integrity metadata. A full, unavailable or replaced store denies Owner Mode. Ordinary requests continue in standard mode. Model prompt and KV-cache policy is configured separately.
Missing or invalid authentication uses standard mode. Valid authentication
selects the installed profile; an authenticated unknown profile is an error.
Return X-Owner-Mode-Effective with the actual selection, including off.
The adapter recognizes these exact HTTP 400 messages for invalid_owner_mode:
| Case | Message |
|---|---|
| Profiles unavailable | this server was started without owner-mode profiles: owner_mode must be "off" |
| Unknown profile | owner_mode names no profile this server loaded |
| Invalid type | owner_mode must be "off" or an owner-mode profile name |
| Invalid scale | owner_mode_scale must be a number in (0, 2] |
The engine error string for scale is matched verbatim. This adapter accepts
only (0, 1]. In Verified contacts mode, only the profiles-unavailable response
causes one retry in standard mode. Always On returns the failure without a
stock retry. Other errors are returned to the caller. Recheck this contract
when upgrading an engine.
Isolate caches by every effective profile, scale and model-revision dimension, or bypass profile-mode cache reads and writes. Standard requests must not reuse profile-mode cache entries or transforms. The engine must validate the profile format and reject unsupported extensions before advertising support.
An in-memory profile index is a sufficient cache discriminator only while the loaded model and profile set stay immutable. Reloading either must discard those entries, or use new keys that include the changed model/profile content. Keys must also distinguish the effective strength. A profile name alone is insufficient.
An effective-profile header reports selection; it does not itself establish numerical correctness or cache isolation. Engine implementations must test those properties independently. Model profiles never grant tool permissions.
Backend manifests declare maximum_rank, the largest projection-basis rank
their loader and runtime accept. Setup checks all operators, including shared
operators in grouped profiles and the global rank of writer-group profiles,
before preparing an engine. The declaration does not extend a backend's model,
site or transform support and is separate from its recorded GPU qualification.
An adapter that supports partial profiles must leave omitted sites unchanged. Accepting a grouped profile must not imply support for every operator family or rank: unsupported requirements, targets and ranks must fail before activation.
The DeepSeek starter uses format: "flashalign.native_wo_b.v1" and
transform: "native_matrix_override_v1". It is separate from an output
projection. Its manifest declares the model revision, tensor encodings,
dimensions, payload checksum, and stock and edited tensor-partition checksums.
The runtime validates these before activation.
This operator keeps stock and profile matrices in immutable bindings. An authenticated owner row uses the profile binding at the declared WO-B sites; an off row uses the stock binding. Selection happens before tensor-parallel reduction and residual mixing. Mixed batches must route rows independently, including during speculative verification and graph replay.
The current native profile accepts strength 1.0 only. A loader must reject other strengths or unsupported requirements rather than interpolate, convert the artifact to a projection, or silently serve a different operator. Its cache identity includes the profile content digest. See the native vLLM backend and TensorFold setup for the exact checkpoint and runtime limits. A TensorFold 1.2.0 bundle declares its compatible EXL3 serving base separately from the profile's original target model. It selects the same edited FP8 tensors without converting the profile.
A profile may declare transform: "weighted_output_projection_v1" and
requires: ["per_direction_strength"]. Each site must have scale: 1 and
these tensors in profile.safetensors:
| Tensor key | Type and shape | Validation |
|---|---|---|
L{layer}.{site}.Q |
FP32 [rank, hidden_size] |
Finite, orthonormal rows |
L{layer}.{site}.strengths |
FP32 [rank] |
Finite, strictly positive values |
For a row batch y of the named matrix's unmodified outputs, apply the operator
before residual addition. With request strength alpha = owner_mode_scale:
y_new = y - alpha * ((y @ Q.T) * strengths) @ QEach direction keeps its own strength. A loader must reject a non-unit site
scale, missing or invalid tensors, or unsupported transform and requires
values. Engines must explicitly support this weighted operator; accepting the
file schema does not establish runtime support. Do not average the strengths or
silently use an ordinary unweighted projection. Standard-mode rows bypass all
transform arithmetic, and mixed batches modify only the authenticated rows.
An engine kit that starts resources outside its process tree declares a
shutdown helper in backend.json:
{"external_cleanup": {"schema": "flashalign.external-cleanup.v1", "script": "cleanup.py", "timeout_seconds": 35}}Setup pins the staged helper's checksum. After reaping managed processes, the
caller runs that helper with --state-dir and --deadline-monotonic, within
the deployment's total shutdown budget. The helper must verify resource
ownership and return JSON with the same schema, status: "stopped" and
external_resources_stopped: true. An unknown resource state, timeout or failed
check leaves shutdown incomplete and preserves its recovery records. A
deployment that declares an external runtime without this contract must be
prepared again before managed shutdown can confirm success.
The native Docker helper checks the daemon identity, container ID, image and ownership labels recorded by the launcher. An unresolved creation intent does not establish that no container exists. Other engines can supply their own helper using the same contract; host-only engines need no helper.
GET /v1/flashalign/status returns schema: "flashalign.runtime-status.v1".
Use a new instance_id when the serving process restarts. generation identifies
the active profile generation or alias epoch. Counters cover all requests in
that engine instance, including stock requests; they are not per-profile
benchmarks. A client must discard deltas across instance, model or generation
changes and when a counter decreases.
Every implementation of this status schema includes these counter fields.
Use JSON null when a measurement is unavailable; never substitute zero:
| Field | Meaning |
|---|---|
output_tokens_total |
Cumulative generated output tokens reported by the engine's metrics collector, including reasoning tokens when counted by that collector |
ttft_seconds_sum |
Cumulative seconds to first token for requests observed by the engine's first-token histogram |
ttft_seconds_count |
Number of observations contributing to ttft_seconds_sum |
Clients treat a missing counter field as unavailable.
Counters must be finite and nonnegative; token and observation counts must be
integers. Return the TTFT sum and count together, or both as null. A real
zero is distinct from a missing metric. Output tokens per second is the token
counter delta divided by elapsed time. Mean TTFT is the sum delta divided by
the observation-count delta. A zero count delta has no mean.
The vLLM adapters read existing in-process vllm:generation_tokens_total and
vllm:time_to_first_token_seconds_{sum,count} samples. SGLang uses the matching
sglang: names, reading generation tokens from the tensor-parallel leader when
it reports one, otherwise from the API process, and first-token observations
from the API process. Label series within a collector
are summed; replicated workers are not summed. Collection is optional: if the
metrics package or samples are absent, status reports null. No metrics
package is imported or installed by the status reader. The llama.cpp adapter
currently reports these counters as unavailable.
Status may also include graph_mode, expected_sites, installed_sites,
exercised_sites, active_requests, and gpu_memory_bytes. The memory field
is allocated device memory as reported by the adapter, not an admission budget.
These fields describe engine state and do not attest numerical correctness.