Skip to content

Commit

Permalink
[Docs] fix: update the parameters instruction in readme (#548)
Browse files Browse the repository at this point in the history
fix: update the parameters instruction in readme

Co-authored-by: chenyu.jiang <[email protected]>
  • Loading branch information
scarlet25151 and chenyu.jiang authored Jan 2, 2025
1 parent 43503f5 commit 3491c2d
Showing 1 changed file with 12 additions and 1 deletion.
13 changes: 12 additions & 1 deletion benchmarks/generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,25 @@ The file would be stored under `output` folder based on the name of different pa

```shell
export SUMMARY_FILE=${PATH_TO_SUMMARY_FILE}
python workload_generator.py --prompt-file $SHAREGPT_FILE_PATH --num-prompts 100 --interval-ms 1000 --duration-ms 600000 --trace-type internal --trace-file "$SUMMARY_FILE" --model "Qwen/Qwen2.5-Coder-7B-Instruct" --output-dir "output"
python workload_generator.py --prompt-file $SHAREGPT_FILE_PATH --num-prompts 100 --interval-ms 1000 --duration-ms 600000 --trace-type internal --traffic-file "$SUMMARY_FILE" --model "Qwen/Qwen2.5-Coder-7B-Instruct" --output-dir "output"
```

This generator assumes trace file to be in the following format
```
"Time","Total","Success","4xx Error"
2024-10-1 00:00:00,100,99,1
```
### Indicate the length of prompt/completion
In this case, you can also indicate the request's prompt length by the `--prompt-len-file` config, or the output length by the `--completion-len-file`,
based on the parameters, the generator will select the proper length in the prompt_file to simulate the length of the real flow's load.

The format of the file should follow the table head format and have the **exact same row length** as the traffic file
```
P50,P70,P99
2000,4000,10000
...
2000,4000,10000(same row size with traffic file)
```

This generator generate workload file (in .json format) under `output` folder. The file would look like the following:
```
Expand Down

0 comments on commit 3491c2d

Please sign in to comment.