Skip to content

Commit 7b7728c

Browse files
Fix vLLM CPU initialize engine issue for DeepSeek models (#1762)
Signed-off-by: lvliang-intel <[email protected]>
1 parent 6917d5b commit 7b7728c

File tree

1 file changed

+1
-0
lines changed

1 file changed

+1
-0
lines changed

ChatQnA/docker_compose/intel/cpu/xeon/compose.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,7 @@ services:
9696
HF_TOKEN: ${HUGGINGFACEHUB_API_TOKEN}
9797
LLM_MODEL_ID: ${LLM_MODEL_ID}
9898
VLLM_TORCH_PROFILER_DIR: "/mnt"
99+
VLLM_CPU_KVCACHE_SPACE: 40
99100
healthcheck:
100101
test: ["CMD-SHELL", "curl -f http://$host_ip:9009/health || exit 1"]
101102
interval: 10s

0 commit comments

Comments
 (0)