Skip to content

error parsing results in genai-perf when sending to dynamo workload #840

@jeffnvidia

Description

@jeffnvidia

I sometimes get this error (happens at random moments when parsing the results) :

json.decoder.JSONDecodeError: Expecting ':' delimiter: line 1 column 414 (char 413)

Here is my command :
genai-perf profile -m deepseek-ai/DeepSeek-R1-Distill-Llama-8B --url dgx-gaia-10:8000 --endpoint-type chat --streaming --endpoint v1/chat/completions --random-seed 123 --synthetic-input-tokens-stddev 0 --output-tokens-stddev 0 --synthetic-input-tokens-mean 2000 --output-tokens-mean 150 --profile-export-file my_profile_export.json --artifact-dir artifacts/ --concurrency 5000 --request-count 5000 --num-dataset-entries 768 --tokenizer deepseek-ai/DeepSeek-R1-Distill-Llama-8B --extra-inputs ignore_eos:true --extra-inputs max_tokens:150 --extra-inputs min_tokens:150 --extra-inputs "{"nvext":{"ignore_eos":true}}" -- -v --max-threads 256 -H 'Authorization: Bearer NOT USED' -H 'Accept: text/event-stream'

I'm running on dynamo disagg vllm_v1 on 2P2D

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions