Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Misc][WIP]Add benchmark scripts #74

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

Potabk
Copy link
Contributor

@Potabk Potabk commented Feb 17, 2025

What this PR does / why we need it?

The purpose of this PR is to add benchmark scripts for npu, developers can easily run performance tests on their own machines with one line of code .

How to use

Prerequisites

  • Please make sure that you have vllm and vllm-ascned installed and npu environment,cause these scripts are specially prepared for npu devices
  • To speed up the script, you can download the models and datasets to your local computer in advance and change the path in the json file in the .elastic/nightly-benchmarks/tests folder

Run benchmarks

these scripts can automatically conduct performance testing of serving, through and latency, run the following command:

cd vllm-ascend
bash .elastic/nightly-benchmarks/scripts/run-performance-benchmarks.sh

once the script is finished, you can view the result files in the benchmarks/results folder. and the results may looks like below:

|-- latency_llama8B_tp1.json
|-- serving_llama8B_tp1_sharegpt_qps_1.json
|-- serving_llama8B_tp1_sharegpt_qps_16.json
|-- serving_llama8B_tp1_sharegpt_qps_4.json
|-- serving_llama8B_tp1_sharegpt_qps_inf.json
|-- throughput_llama8B_tp1.json

Does this PR introduce any user-facing change?

How was this patch tested?

@Potabk Potabk changed the title [Misc]dd benchmark scripts [Misc]Add benchmark scripts Feb 17, 2025
@Potabk Potabk changed the title [Misc]Add benchmark scripts [Misc][WIP]Add benchmark scripts Feb 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant