Skip to content

Add Queue Utilization & Cost charts into vLLM CI dashboard #7380

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add Queue Utilization & Cost charts into vLLM CI dashboard #7380

wants to merge 1 commit into from

Conversation

@hl475
Copy link

@hl475 hl475 commented Oct 18, 2025 •
edited
Loading

This PR adds two charts into https://hud.pytorch.org/metrics/vllm regarding vLLM CI queue util and costs. In specific, per buildkite job, we want to track

  • track 3 queues gpu_1_queue, gpu_4_queue, and cpu_queue_premerge_us_east_1 queue time
  • costs running on gpu_1_queue and gpu_4_queue
  • a line chart to show the daily average trend

Preview - https://torchci-git-vllmqueuestatus-fbopensource.vercel.app/metrics/vllm

@meta-cla meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 18, 2025
@vercel
Copy link

vercel bot commented Oct 18, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Updated (UTC)
torchci Ready Ready Preview Oct 27, 2025 4:56pm

@hl475 hl475 force-pushed the vllm_queue_status branch from 14da1e2 to c417534 Compare October 19, 2025 00:14
@hl475 hl475 force-pushed the vllm_queue_status branch from c417534 to 469ebf4 Compare October 19, 2025 00:23
@hl475 hl475 force-pushed the vllm_queue_status branch from 469ebf4 to 237e504 Compare October 19, 2025 00:28
@hl475 hl475 force-pushed the vllm_queue_status branch from 237e504 to f82793f Compare October 19, 2025 00:48
@hl475 hl475 force-pushed the vllm_queue_status branch from f82793f to 217f588 Compare October 19, 2025 01:10
@hl475 hl475 force-pushed the vllm_queue_status branch from 217f588 to 7744fa5 Compare October 19, 2025 06:40
@hl475 hl475 force-pushed the vllm_queue_status branch from 7744fa5 to 06ce83d Compare October 19, 2025 06:57
@hl475 hl475 force-pushed the vllm_queue_status branch from 06ce83d to b9e525b Compare October 19, 2025 07:21
@hl475 hl475 force-pushed the vllm_queue_status branch from b9e525b to 9c03307 Compare October 19, 2025 07:35
@hl475 hl475 force-pushed the vllm_queue_status branch from 9c03307 to 619b852 Compare October 19, 2025 20:22
@hl475 hl475 force-pushed the vllm_queue_status branch from 619b852 to b8c8bb8 Compare October 19, 2025 20:45
@hl475 hl475 force-pushed the vllm_queue_status branch from b8c8bb8 to d7e3dc3 Compare October 19, 2025 21:27
@hl475 hl475 force-pushed the vllm_queue_status branch from d7e3dc3 to e125051 Compare October 19, 2025 21:48
@hl475 hl475 force-pushed the vllm_queue_status branch from e125051 to cdf7272 Compare October 19, 2025 22:10
@hl475 hl475 force-pushed the vllm_queue_status branch from 7100680 to 0d35931 Compare October 19, 2025 23:56
@hl475 hl475 force-pushed the vllm_queue_status branch from 0d35931 to 01c97ad Compare October 20, 2025 00:05
@hl475 hl475 force-pushed the vllm_queue_status branch from 01c97ad to 4966a1c Compare October 20, 2025 00:08
@hl475 hl475 force-pushed the vllm_queue_status branch from 4966a1c to c72a764 Compare October 20, 2025 00:14
@hl475 hl475 force-pushed the vllm_queue_status branch from c72a764 to 175acc0 Compare October 20, 2025 00:21
@hl475 hl475 force-pushed the vllm_queue_status branch from 175acc0 to 1c16cb7 Compare October 20, 2025 00:28
@hl475 hl475 force-pushed the vllm_queue_status branch from 1c16cb7 to 6235c82 Compare October 20, 2025 00:43
@hl475 hl475 changed the title [WIP] vllm_queue_status [WIP] Add Queue Utilization & Cost charts into vLLM CI dashboard Oct 20, 2025
@rzabarazesh
Copy link
Contributor

I think the "main only" toggle is not functioning properly.

@hl475 hl475 force-pushed the vllm_queue_status branch from 6235c82 to f2b9b5e Compare October 21, 2025 04:38
@hl475 hl475 force-pushed the vllm_queue_status branch from f2b9b5e to 35a1430 Compare October 21, 2025 04:45
@hl475 hl475 force-pushed the vllm_queue_status branch from 35a1430 to 29a9856 Compare October 21, 2025 04:51
@hl475
vllm_queue_status
38335f4 
Signed-off-by: Huamin Li <[email protected]>
@hl475 hl475 force-pushed the vllm_queue_status branch from 29a9856 to 38335f4 Compare October 27, 2025 16:54
@hl475 hl475 changed the title [WIP] Add Queue Utilization & Cost charts into vLLM CI dashboard Add Queue Utilization & Cost charts into vLLM CI dashboard Oct 27, 2025
@hl475 hl475 marked this pull request as ready for review October 27, 2025 17:12
@hl475 hl475 requested review from huydhn and rzabarazesh October 27, 2025 17:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@huydhn huydhn Awaiting requested review from huydhn

@rzabarazesh rzabarazesh Awaiting requested review from rzabarazesh

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@hl475 @rzabarazesh