File tree Expand file tree Collapse file tree 1 file changed +3
-2
lines changed Expand file tree Collapse file tree 1 file changed +3
-2
lines changed Original file line number Diff line number Diff line change @@ -272,6 +272,7 @@ See list of parameters at [llm-d-kv-cache-manager/docs/configuration.md](https:/
272272
273273Note that in most cases you will only need to set :
274274- Hugging Face token for the `tokenizersPoolConfig` or the `tokenizersCacheDir` to a mounted directory containing the tokenizers.
275+ - For the HuggingFace token, the inference-scheduler also accepts the environment variable `HF_TOKEN` - this is the practical option for security.
275276- IMPORTANT : Token processor's block-size and hash-seed to match those used in the vLLM deployment.
276277- KVBlockIndex metrics to true if you wish to enable metrics for the KV-Block Index (admissions, evictions, lookups and hits).
277278
@@ -285,7 +286,7 @@ plugins:
285286 blockSize: 64
286287 hashSeed: "12345"
287288 tokenizersPoolConfig:
288- huggingFaceToken: your_hf_token_here
289+ huggingFaceToken: your_hf_token_here # automatically set by ` HF_TOKEN` environment variable
289290 kvBlockIndexConfig :
290291 enableMetrics : true
291292` ` `
@@ -313,7 +314,7 @@ plugins:
313314 enableMetrics : true
314315 tokenizersPoolConfig :
315316 workersCount : 8
316- huggingFaceToken: your_hf_token_here
317+ huggingFaceToken : your_hf_token_here # automatically set by `HF_TOKEN` environment variable
317318 tokenizersCacheDir : /tmp/tokenizers
318319` ` `
319320
You can’t perform that action at this time.
0 commit comments