feat: resolve oci:// model ids via llmman serve

[ericcurtin]

What does this PR do?

Adds an oci:// scheme so a model published as a CNCF ModelPack artifact can be used anywhere a Hugging Face repo id can:

text-embeddings-router --model-id oci://ghcr.io/org/model:tag

Model distribution is increasingly moving to OCI registries -- the same registries, credentials, mirroring and air-gap tooling a deployment already uses for container images. This is usually easier to run air-gapped than reaching the Hub.

Implementation

One branch in run() in router/src/lib.rs, alongside the existing local-directory check:

let (model_root, api_repo) = if oci::is_oci_ref(&model_id) {
    (oci::from_oci(&model_id).await?, None)
} else if model_id_path.exists() && model_id_path.is_dir() {
    ...

The extracted directory is then loaded exactly like any other local model -- config.json, get_backend_model_type, tokenizer and dense-module discovery all operate on it unchanged. api_repo is None, as it already is for a local model, since there is no Hub repo behind an OCI artifact; the 1_Dense / dense-path lookups fall back to what is on disk.

New router/src/oci.rs delegates acquisition to a running llmman serve rather than reimplementing registry handling: llmman already implements the ModelPack media types, registry auth, resumable blob download and a content-addressed store. Two pieces are needed, because the daemon deliberately exposes no local path:

• GET /api/version probes reachability and identity -- a server answering without a version field is reported as "not an llmman daemon", worth distinguishing from nothing listening.
• POST /api/pull streams newline-delimited JSON so a multi-gigabyte fetch is not silent. An error arrives in-band at HTTP 200, and a stream that simply ends without success is also a failure -- both are errors, not a completed pull.
• llmman resolve --no-pull then reports where the bytes landed. --no-pull guarantees it only reports on what /api/pull already fetched, keeping the daemon the only thing that touches the network.

LLMMAN_HOST is honoured with the same parsing llmman's own clients use, including rewriting a wildcard bind (0.0.0.0, [::]) to loopback by value rather than spelling, so an expanded IPv6 form is caught too. A pull needs both the daemon reachable and the binary on PATH (or TEI_LLMMAN_BIN); each missing piece has its own actionable error, and neither is required unless an oci:// id is used.

Dependency changes

No new crates. reqwest (already a router dependency) gains json + stream; tokio gains process; tempfile is added as a dev-dependency for the tests.

Design notes

• Explicit scheme, no sniffing. A bare registry/name:tag is indistinguishable from a Hugging Face repo id (BAAI/bge-large-en-v1.5); guessing would silently hijack existing --model-id org/model deployments. Every other id shape reaches exactly the branch it did before.
• Tolerant parsing. A non-JSON diagnostic in the NDJSON stream is skipped rather than aborting a pull still in progress; the last non-empty line of resolve stdout is used; unknown JSON fields are ignored so the contract can grow.

Testing

14 unit tests in router::oci, all executed and passing:

• scheme detection incl. case-insensitivity (OCI://) and the empty-scheme edge case
• that a bare HF repo id, a local path, and s3:// / hf:// are not claimed -- the regression that matters most
• strip_scheme round-trips
• endpoint resolution: default, every LLMMAN_HOST form (bare host, host:port, scheme prefix, trailing path, quoted), and wildcard-to-loopback rewriting incl. [0:0:0:0:0:0:0:0]
• the NDJSON pull protocol at line level: in-band error, success, byte progress, tolerance of a non-JSON diagnostic and blank lines
• the resolve contract: documented shape, leaked-diagnostic tolerance, unknown-field tolerance, and seven malformed-output cases
• empty reference rejected without contacting the daemon

cargo test -p text-embeddings-router --lib oci
test result: ok. 14 passed; 0 failed; 0 ignored

• cargo check -p text-embeddings-router clean
• cargo clippy -p text-embeddings-router --lib --tests -- no warnings
• rustfmt --check router/src/oci.rs clean. Note cargo fmt cannot run repo-wide here: it fails on router/src/grpc/pb.rs (generated at build time, absent in a fresh checkout) on main too, unrelated to this change.

Not verified here, flagged rather than implied: no end-to-end serve against a live llmman serve backed by a real registry. The HTTP client is covered at the line-protocol level rather than against a stub server.

Before submitting

• Did you read the contributor guideline?
• Was this discussed/approved via a Github issue or the forum? N/A -- small additive feature.
• Did you make sure to update the documentation with your changes? Covered in the PR body; happy to add to docs/ if wanted.
• Did you write any new necessary tests?

Disclosure: written with AI assistance.