[FEATURE] Estimate VRAM for local safetensors files

[almeidava93]

It would be nice to make the same estimation for local models with .safetensors files. For example, mmBERT-base does not have a .safetensor file on HF, but I have a fine-tuned version of it that has safetensors. I would like to do the estimation for it.

I can work on the PR if you find useful.

[vrdn-23]

+1
Could be similar to what is trying to be done here: huggingface/huggingface_hub#3623

[alvarobartt]

Yes, @almeidava93 100%.

Once GGUF support is included I'll take on a small refactor to make things clearer and include local file support (+ cache resolution > HTTP requests when applicable), most likely in a couple of weeks as per the current plans. Otherwise, feel free to take a shot at it, but as I just said it might be tricky as we're still in the process of adding GGUF support, see #22 and #25

[vrdn-23]

Looks this got closed accidentally @alvarobartt ? Or is this being supported in the new release?

[alvarobartt]

Looks this got closed accidentally @alvarobartt ? Or is this being supported in the new release?

Definitely not an accident, @diegovelilla added support for it in the soon to be released hf-mem but already available on main!

[vrdn-23]

Umm I think maybe this was misunderstood because I looked through the codebase and couldn't find where the path was for calculating local files. Could you maybe check again if this is actually implemented in that PR? @alvarobartt

[alvarobartt]

Umm I think maybe this was misunderstood because I looked through the codebase and couldn't find where the path was for calculating local files. Could you maybe check again if this is actually implemented in that PR? @alvarobartt

Arf you're right, I misread the issue and thought this was the GGUF support one! Indeed it's not supported yes, I'll reopen and create the PR this week, apologies and thanks for the heads up @vrdn-23 🙏