Skip to content

fix: forward cache_dir to ONNX and OpenVINO model loading#3691

Open
AtharvaJaiswal005 wants to merge 1 commit intohuggingface:mainfrom
AtharvaJaiswal005:fix/onnx-cache-dir-forwarding
Open

fix: forward cache_dir to ONNX and OpenVINO model loading#3691
AtharvaJaiswal005 wants to merge 1 commit intohuggingface:mainfrom
AtharvaJaiswal005:fix/onnx-cache-dir-forwarding

Conversation

@AtharvaJaiswal005
Copy link

Summary

  • Forward cache_dir from Transformer._load_model() to load_onnx_model() and load_openvino_model()
  • Forward cache_dir to the underlying model_cls.from_pretrained() calls in both functions
  • Fixes ONNX/OpenVINO backend failing with ValueError when cache_folder is specified alongside local_files_only=True

Root Cause

When using backend="onnx" (or "openvino") with a custom cache_folder, the cache_dir parameter was not forwarded through the loading chain:

  1. Transformer._load_model() receives cache_dir but did not pass it to load_onnx_model() / load_openvino_model() (unlike the torch backend which passes cache_dir to AutoModel.from_pretrained())
  2. load_onnx_model() / load_openvino_model() did not accept or forward cache_dir to model_cls.from_pretrained()

This meant the ONNX/OpenVINO model loader had no knowledge of where cached files were stored, causing it to fail when local_files_only=True.

Reproduction

from sentence_transformers import SentenceTransformer

# Step 1: Download to cache (works)
model = SentenceTransformer(
    "sentence-transformers/all-MiniLM-L6-v2",
    cache_folder="./model_cache/",
    backend="onnx",
    local_files_only=False,
)

# Step 2: Load from cache (fails before fix)
model = SentenceTransformer(
    "sentence-transformers/all-MiniLM-L6-v2",
    cache_folder="./model_cache/",
    backend="onnx",
    local_files_only=True,
)

Changes

  • sentence_transformers/backend/load.py: Added cache_dir parameter to load_onnx_model() and load_openvino_model(), forwarded to from_pretrained()
  • sentence_transformers/models/Transformer.py: Pass cache_dir when calling load_onnx_model() and load_openvino_model() in _load_model()

Fixes #3562

When cache_folder is specified with backend="onnx" or backend="openvino",
the cache_dir was not forwarded to load_onnx_model/load_openvino_model or
to the underlying from_pretrained call. This caused a ValueError when
loading cached models with local_files_only=True because the model files
could not be found without the cache directory path.

Fixes huggingface#3562
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ONNX backend does not work when cache_folder is specified and local_files_only is True

1 participant