Background
Domux currently does not recognize GGUF format models used by Ollama. Users cannot import local GGUF weights or connect to the Ollama server directly, requiring extra model format conversion steps before inference.
Acceptance Criteria
- System can parse GGUF v2 & v3 files correctly, compatible with Q4_0/Q4_K_M/Q5_K_M/Q6_K/Q8_0 quantization.
- UI provides entry to upload local
.gguf files, auto extract model metadata without manual configuration.
- Integrate Ollama REST API: connect local/remote Ollama instance, pull & launch official models in one click.
- Optimize weight loader to cut peak memory usage during large GGUF file loading, with loading progress display.
- Separate storage logic for GGUF and Safetensors models, refactor model management module.
- Complete user documentation for GGUF local import and Ollama integration.
Background
Domux currently does not recognize GGUF format models used by Ollama. Users cannot import local GGUF weights or connect to the Ollama server directly, requiring extra model format conversion steps before inference.
Acceptance Criteria
.gguffiles, auto extract model metadata without manual configuration.