feat(sdk): pull models from Docker Hub's ai/* namespace#1165
Open
Eric Curtin (ericcurtin) wants to merge 1 commit into
Open
feat(sdk): pull models from Docker Hub's ai/* namespace#1165Eric Curtin (ericcurtin) wants to merge 1 commit into
Eric Curtin (ericcurtin) wants to merge 1 commit into
Conversation
Contributor
|
RemiliaForever (@RemiliaForever) Mengsheng Wu (@mengshengwu) Plz help review! Thanks for your support! Eric Curtin (@ericcurtin) |
Add a DockerHubSource that speaks the Docker Registry HTTP API V2 to resolve and download GGUF model artifacts published under https://hub.docker.com/u/ai (the same protocol `docker model pull ai/<repo>` uses under the hood), mirroring docker/model-runner's pkg/distribution package: anonymous bearer token exchange, a content-negotiated manifest GET, and a small config blob GET that carries the GGUF quantization tag and tells us whether the model is a VLM (an mmproj layer present) — all before any weight bytes stream, so this fits the existing plan-then-fetch ModelSource contract used by the HuggingFace and AI Hub sources. Only the Docker proprietary format is supported (v0.1 single-layer- per-format with positional naming, and v0.2 layer-per-file with `org.cncf.model.filepath` annotations); CNCF ModelPack artifacts and non-GGUF weights (safetensors/DDUF) are rejected with an actionable error rather than silently mishandled. - crates/core: new source::dockerhub module + PullIntent::DockerHub; mapping::is_docker_hub_reference / docker_hub_repo_from_name so a `docker.io/ai/gemma3`-style reference auto-routes without an explicit hub flag (a bare "ai/gemma3" is never auto-routed, since that shape is indistinguishable from a HuggingFace repo). - crates/ffi + geniex_model.h: new GENIEX_HUB_DOCKER hub selector. `quant` doubles as the Docker tag/digest for this hub (empty means "latest") since there's no GGUF-quant filtering step to reuse it for. - bindings/go + cli: `--model-hub docker`, plus a case-preserving name/tag split for Docker references so a mixed-case tag isn't mistakenly upper-cased the way a GGUF quantization label is. Signed-off-by: Eric Curtin <eric.curtin@docker.com>
f2685eb to
57fc66d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds support for pulling GGUF model artifacts published under
hub.docker.com/u/ai (e.g.
ai/gemma3,ai/smollm2), the same protocoldocker model pull ai/<repo>uses.The example implementation this follows is
docker/model-runner'spkg/distributionpackage.sdk/model-manager/crates/core/src/source/dockerhub.rs(new): aModelSourcethat talks the plain Docker Registry HTTP API V2 —anonymous bearer token exchange, a content-negotiated manifest GET,
and a small config blob GET (carries the GGUF quantization tag and
tells us whether the model is a VLM via an
mmprojlayer) — allbefore any weight bytes stream, fitting the existing
plan-then-fetch contract the HuggingFace and AI Hub sources already
use.
(single-layer-per-format, positional naming —
model.gguformodel-%05d-of-%05d.ggufshards, matchingdocker/model-runner'sunpackLegacyexactly) and "v0.2"(layer-per-file with
org.cncf.model.filepathannotations).with an actionable error instead of mishandling them (not yet
supported).
mapping.rs:is_docker_hub_reference/docker_hub_repo_from_nameso
docker.io/ai/gemma3(orindex.docker.io/…,https://hub.docker.com/r/…) auto-routes to Docker Hub without anexplicit hub flag. A bare
ai/gemma3is deliberately notauto-routed — that shape is indistinguishable from a HuggingFace
repo of the same name, so it needs
--model-hub docker(mirrorshow HF/AI Hub ambiguity is already handled for
qualcomm/*).pull.rs: newPullIntent::DockerHub { repo, reference }.crates/ffi/src/pull.rs,include/geniex_model.h): newGENIEX_HUB_DOCKERhub selector. The existingquantfield doublesas the Docker tag/digest for this hub (empty →
"latest"), sincethere's no GGUF-quant-filter step to reuse it for otherwise.
--model-hub docker, plusSplitNamePrecisionCaseSensitive/IsDockerHubReferenceso aDocker tag isn't mistakenly upper-cased the way a GGUF
quantization label is (
ai/gemma3:latestmust not become:LATEST).geniex infer docker.io/ai/gemma3.Try it
geniex pull docker.io/ai/gemma3 geniex infer ai/gemma3 --model-hub docker geniex infer docker.io/ai/gemma3:F16 # explicit tagTest plan
cargo test --workspaceinsdk/model-manager(existing 151 +5 executor/proxy/pull-resume integration suites, plus 7 new
source::dockerhubunit tests covering: single-GGUF v0.1layout, VLM detection via
mmprojlayer, multi-shard GGUFnaming, v0.2 filepath-annotation naming, safetensors-only
rejection, and 404 →
HubModelNotFound).cargo clippy --workspace --all-targets— no new warnings.cargo fmt --check— clean.mapping::unit tests foris_docker_hub_reference/docker_hub_repo_from_nameprefix handling.go build ./...forbindings/go+cli— not verified inthis environment (requires the full CMake-staged
sdk/pkg-geniexheader/lib tree from/build); changes areadditive to the existing enum/struct and were reviewed by hand
+
gofmt.