Unable to install/fetch Fined-tuned Whisper model in private repo using WhisperKit?

[velairi]

Hi WhisperKit team,

I'm building an iOS app that uses a fine-tuned Whisper model hosted in a private Hugging Face repository, but I'm encountering authorization issues when trying to load the model programmatically.
Already followed this: https://github.com/argmaxinc/whisperkittools?tab=readme-ov-file#-model-generation-apple

Issue:
When attempting to load my model teamXYZ/custom-whisper-model-madeForWhisperKit using a Hugging Face token with full repository access, I receive an authorization error despite the token working correctly in other contexts.

What works:
The model functions perfectly when using the CLI with local files:

swift run whisperkit-cli transcribe --model-path "Models/whisperkit-coreml/custom-whisper-model-madeForWhisperKit" --audio-path "/Users/username/datasets/US-AR7-002_1.wav"

What doesn't work:
Programmatic access to the private Hugging Face repository fails with authorization errors.

Current implementation:

func loadWhisperModel() async -> String? {
    guard let url = URL(string: "https://huggingface.co/teamXYZ/custom-whisper-model-madeForWhisperKit") else {
        return nil
    }
    
    var request = URLRequest(url: url, timeoutInterval: Double.infinity)
    request.addValue("Bearer <my_hf_token>", forHTTPHeaderField: "Authorization")
    request.addValue("application/json", forHTTPHeaderField: "Content-Type")
    request.httpMethod = "GET"
    
    do {
        let (data, response) = try await URLSession.shared.data(for: request)
        print(String(data: data, encoding: .utf8))
    } catch {
        print("Error during API request: \(error)")
        return nil
    }
    return ""
}

Questions:

1. Does WhisperKit have built-in support for authenticating with private Hugging Face repositories?
2. Should I be using a different API endpoint or authentication method?
3. Is there a recommended way to load private models in WhisperKit for iOS apps?

Any guidance would be greatly appreciated!

[atiorh]

Hello! For private repos, you can simply point to the repo with modelRepo and if you follow the the same folder structure as the original whisperkit-coreml HF model repo, then you should be able to use this private repo.

[velairi]

Right, we've tried something like this and receiving an authorization error (despite adding the same token via HuggingFace CLI and successfully downloading the model)

    let config = WhisperKitConfig(downloadBase: URL(string: "https://huggingface.co/"),
                                  modelRepo: "teamXYZ/custom-whisper-model-madeForWhisperKit",
                                  modelToken: "<my token with full write access>")

[a2they]

hi @velairi, the downloadBase is the base directory for downloads (defaults to Documents/huggingface). You don't need to specify https://huggingface.co/ as it is the default endpoint.

[atiorh]

Please keep in mind that Hugging Face rate limits private repo access.