Skip to content

Conversation

yannicks1
Copy link
Collaborator

@yannicks1 yannicks1 commented Sep 29, 2025

this change does always compile for the next power of 2 for the batch size.

Copy link

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

tokenizer=model_name,
max_model_len=max(max_model_len, 256),
max_num_seqs=max_num_seqs,
max_num_seqs=max(max_num_seqs, 4),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will leave the test name as-is, which might be confusing (since the test will still say BS 2)

Signed-off-by: Yannick Schnider <[email protected]>
Signed-off-by: Yannick Schnider <[email protected]>
@yannicks1 yannicks1 marked this pull request as ready for review September 29, 2025 22:53
@yannicks1
Copy link
Collaborator Author

bot:test

Copy link
Collaborator

@maxdebayser maxdebayser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with this as long as the tests pass. Just for the record, can you add an explanation either here in the PR or in the code as to why this is required?

@joerunde
Copy link
Collaborator

Just for the record, can you add an explanation either here in the PR or in the code as to why this is required?

+1, can we extend the 🌶️🌶️🌶️ comment right above these changes?

@wallashss
Copy link
Collaborator

Another +1 for the documentation of the solution and question.

If we have this constraint of batch size, should we update platform as well somehow to give feedback to the user?

Signed-off-by: Yannick Schnider <[email protected]>
@yannicks1
Copy link
Collaborator Author

just added some more comments

@yannicks1
Copy link
Collaborator Author

bot:test
TORCH_SENDNN_CACHE_ENABLE=0

@wallashss
Copy link
Collaborator

bot:test

@wallashss wallashss merged commit 9684a82 into main Oct 1, 2025
19 checks passed
@wallashss wallashss deleted the ysc-fix-fp8-test-cb branch October 1, 2025 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants