-
Notifications
You must be signed in to change notification settings - Fork 248
Description
I sometimes get this error (happens at random moments when parsing the results) :
json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 414 (char 413)
Here is my command :
genai-perf profile -m deepseek-ai/DeepSeek-R1-Distill-Llama-8B --url dgx-gaia-10:8000 --endpoint-type chat --streaming --endpoint v1/chat/completions --random-seed 123 --synthetic-input-tokens-stddev 0 --output-tokens-stddev 0 --synthetic-input-tokens-mean 2000 --output-tokens-mean 150 --profile-export-file my_profile_export.json --artifact-dir artifacts/ --concurrency 5000 --request-count 5000 --num-dataset-entries 768 --tokenizer deepseek-ai/DeepSeek-R1-Distill-Llama-8B --extra-inputs ignore_eos:true --extra-inputs max_tokens:150 --extra-inputs min_tokens:150 --extra-inputs "{"nvext":{"ignore_eos":true}}" -- -v --max-threads 256 -H 'Authorization: Bearer NOT USED' -H 'Accept: text/event-stream'
I'm running on dynamo disagg vllm_v1 on 2P2D