local models

[anderbarrena]

I'm trying to merge models from disk instead of using Hugging Face links, but I can't get it to work. I'm encountering the following error: RuntimeError: Can't parse "modelpath"

for instance a config, base model has the path to the model:

models:
- model: meta-llama/Llama-3.1-8B-Instruct
parameters:
density: [1, 0.7, 0.1] # density gradient
weight: 1.0
merge_method: dare_ties
base_model: /xxxx/xxxxx/
parameters:
normalize: true
int8_mask: true
dtype: float16

Do you have any idea how to fix this?

Ander

[David-AU-github]

If the online repo has restrictions (IE must approve, fill in a form etc), you may get this error.
If memory serves, Llama 3.1-Instruct does have this - download a local copy, then access it via your local drive.

[anderbarrena]

Sorry, I probably didn’t explain the issue clearly. I’m having trouble loading a model from a local directory. The error occurs when I try to load the base model specified in the YAML file, which points to a model stored in a local directory /xxx/xxxx/.

[cg123]

Are there + characters in the directory path you're specifying? If so you won't be able to use that path as a raw string, as it conflicts with the base_model_ref+lora_model_ref syntax. If you need to reference a model with a path like that you can do so like this:

base_model:
  model:
    path: /your/path/with+plus+signs/here
#   revision: optional revision
# lora: optional LoRA to apply
#   path: ...
#   revision: ...
parameters: # ... and so on

[anderbarrena]

Yes! That was the cause of the error.
Thanks!!! :)