Skip to content

Conversation

@rafvasq
Copy link
Collaborator

@rafvasq rafvasq commented Aug 8, 2025

Description

  • Cleans up create_text_prompt
  • Adds create_seq_prompt
  • Adds six test cases:
    • 32 prompts with 512 tokens each
    • 16 prompts with 1.5k tokens each
    • 8 prompts with 3k tokens each
    • 4 prompts with 5k tokens each
    • 2 prompts with 9k tokens each
    • 1 prompt with 17k tokens

rafvasq added 2 commits August 8, 2025 15:39
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq requested a review from joerunde August 8, 2025 19:41
@github-actions
Copy link

github-actions bot commented Aug 8, 2025

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq marked this pull request as ready for review August 8, 2025 20:14
@rafvasq rafvasq marked this pull request as draft August 8, 2025 20:17
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq marked this pull request as ready for review August 11, 2025 16:18
@rafvasq rafvasq requested a review from joerunde August 11, 2025 16:18
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq force-pushed the add-long-context-batch-tests branch from efdc43b to a0f95b0 Compare August 13, 2025 14:26
Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq
Copy link
Collaborator Author

rafvasq commented Aug 13, 2025

Test passes on CPU:

tests/e2e/test_spyre_cb.py::test_long_context_batches[TP(1)-eager-ibm-ai-platform/micro-g3.3-8b-instruct-1b] PASSED [ 50%]
tests/e2e/test_spyre_cb.py::test_long_context_batches[TP(1)-eager-ibm-ai-platform/micro-g3.3-8b-instruct-1b-FP8] SKIPPED [100%]

Signed-off-by: Rafael Vasquez <[email protected]>
Copy link
Collaborator

@yannicks1 yannicks1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Final remark: will this (in its current form) run with GHA? if so, I believe we should remove it. On cpu there is nothing new here, and it is expected to work. IMO this is a test for Spyre only and should not be run on cpu for every PR...

Signed-off-by: Rafael Vasquez <[email protected]>
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq requested a review from yannicks1 August 14, 2025 13:09
Copy link
Collaborator

@yannicks1 yannicks1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@rafvasq rafvasq enabled auto-merge (squash) August 14, 2025 14:27
@github-actions github-actions bot added the ready label Aug 14, 2025
@rafvasq rafvasq disabled auto-merge August 14, 2025 15:48
Signed-off-by: Rafael Vasquez <[email protected]>
@rafvasq rafvasq enabled auto-merge (squash) August 14, 2025 15:57
@rafvasq rafvasq merged commit f76c391 into vllm-project:main Aug 14, 2025
27 of 36 checks passed
@rafvasq rafvasq deleted the add-long-context-batch-tests branch August 14, 2025 17:17
@rafvasq rafvasq mentioned this pull request Sep 22, 2025
rafvasq added a commit that referenced this pull request Sep 24, 2025
Re-introducing CB test up to 32k context length

Related to #389,
#365

---------

Signed-off-by: Rafael Vasquez <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants