Skip to content

feat(sdk): pull models from Docker Hub's ai/* namespace#1165

Open
Eric Curtin (ericcurtin) wants to merge 1 commit into
qualcomm:mainfrom
ericcurtin:feat/docker-hub-model-pull
Open

feat(sdk): pull models from Docker Hub's ai/* namespace#1165
Eric Curtin (ericcurtin) wants to merge 1 commit into
qualcomm:mainfrom
ericcurtin:feat/docker-hub-model-pull

Conversation

@ericcurtin

Copy link
Copy Markdown

Summary

Adds support for pulling GGUF model artifacts published under
hub.docker.com/u/ai (e.g. ai/gemma3,
ai/smollm2), the same protocol docker model pull ai/<repo> uses.
The example implementation this follows is
docker/model-runner's
pkg/distribution package.

  • sdk/model-manager/crates/core/src/source/dockerhub.rs (new): a
    ModelSource that talks the plain Docker Registry HTTP API V2 —
    anonymous bearer token exchange, a content-negotiated manifest GET,
    and a small config blob GET (carries the GGUF quantization tag and
    tells us whether the model is a VLM via an mmproj layer) — all
    before any weight bytes stream, fitting the existing
    plan-then-fetch contract the HuggingFace and AI Hub sources already
    use.
    • Supports the Docker proprietary format only: "v0.1"
      (single-layer-per-format, positional naming — model.gguf or
      model-%05d-of-%05d.gguf shards, matching
      docker/model-runner's unpackLegacy exactly) and "v0.2"
      (layer-per-file with org.cncf.model.filepath annotations).
    • Rejects safetensors/DDUF-only images and CNCF ModelPack artifacts
      with an actionable error instead of mishandling them (not yet
      supported).
  • mapping.rs: is_docker_hub_reference / docker_hub_repo_from_name
    so docker.io/ai/gemma3 (or index.docker.io/…,
    https://hub.docker.com/r/…) auto-routes to Docker Hub without an
    explicit hub flag. A bare ai/gemma3 is deliberately not
    auto-routed — that shape is indistinguishable from a HuggingFace
    repo of the same name, so it needs --model-hub docker (mirrors
    how HF/AI Hub ambiguity is already handled for qualcomm/*).
  • pull.rs: new PullIntent::DockerHub { repo, reference }.
  • FFI (crates/ffi/src/pull.rs, include/geniex_model.h): new
    GENIEX_HUB_DOCKER hub selector. The existing quant field doubles
    as the Docker tag/digest for this hub (empty → "latest"), since
    there's no GGUF-quant-filter step to reuse it for otherwise.
  • Go bindings + CLI: --model-hub docker, plus
    SplitNamePrecisionCaseSensitive / IsDockerHubReference so a
    Docker tag isn't mistakenly upper-cased the way a GGUF
    quantization label is (ai/gemma3:latest must not become
    :LATEST).
  • README: one more quickstart line showing
    geniex infer docker.io/ai/gemma3.

Try it

geniex pull docker.io/ai/gemma3
geniex infer ai/gemma3 --model-hub docker
geniex infer docker.io/ai/gemma3:F16   # explicit tag

Test plan

  • cargo test --workspace in sdk/model-manager (existing 151 +
    5 executor/proxy/pull-resume integration suites, plus 7 new
    source::dockerhub unit tests covering: single-GGUF v0.1
    layout, VLM detection via mmproj layer, multi-shard GGUF
    naming, v0.2 filepath-annotation naming, safetensors-only
    rejection, and 404 → HubModelNotFound).
  • cargo clippy --workspace --all-targets — no new warnings.
  • cargo fmt --check — clean.
  • mapping:: unit tests for is_docker_hub_reference /
    docker_hub_repo_from_name prefix handling.
  • go build ./... for bindings/go + cli — not verified in
    this environment (requires the full CMake-staged
    sdk/pkg-geniex header/lib tree from /build); changes are
    additive to the existing enum/struct and were reviewed by hand
    + gofmt.

@alexchen4ai

Copy link
Copy Markdown
Contributor

RemiliaForever (@RemiliaForever) Mengsheng Wu (@mengshengwu) Plz help review! Thanks for your support! Eric Curtin (@ericcurtin)

Add a DockerHubSource that speaks the Docker Registry HTTP API V2 to
resolve and download GGUF model artifacts published under
https://hub.docker.com/u/ai (the same protocol `docker model pull
ai/<repo>` uses under the hood), mirroring docker/model-runner's
pkg/distribution package: anonymous bearer token exchange, a
content-negotiated manifest GET, and a small config blob GET that
carries the GGUF quantization tag and tells us whether the model is a
VLM (an mmproj layer present) — all before any weight bytes stream, so
this fits the existing plan-then-fetch ModelSource contract used by
the HuggingFace and AI Hub sources.

Only the Docker proprietary format is supported (v0.1 single-layer-
per-format with positional naming, and v0.2 layer-per-file with
`org.cncf.model.filepath` annotations); CNCF ModelPack artifacts and
non-GGUF weights (safetensors/DDUF) are rejected with an actionable
error rather than silently mishandled.

- crates/core: new source::dockerhub module + PullIntent::DockerHub;
  mapping::is_docker_hub_reference / docker_hub_repo_from_name so a
  `docker.io/ai/gemma3`-style reference auto-routes without an
  explicit hub flag (a bare "ai/gemma3" is never auto-routed, since
  that shape is indistinguishable from a HuggingFace repo).
- crates/ffi + geniex_model.h: new GENIEX_HUB_DOCKER hub selector.
  `quant` doubles as the Docker tag/digest for this hub (empty means
  "latest") since there's no GGUF-quant filtering step to reuse it
  for.
- bindings/go + cli: `--model-hub docker`, plus a case-preserving
  name/tag split for Docker references so a mixed-case tag isn't
  mistakenly upper-cased the way a GGUF quantization label is.

Signed-off-by: Eric Curtin <eric.curtin@docker.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants