Skip to content

Conversation

maxdebayser
Copy link
Collaborator

Description

The token type ids in upstream vllm are passed to the model runner in the pooling params. This PR fixes the preparation of token type inputs and also fixes the tensor sizes and dimensions to work both with the transformer modeling code and the vllm pooler code.

Copy link

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

@maxdebayser
Copy link
Collaborator Author

bot:test

Copy link
Collaborator

@joerunde joerunde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌶️ , thanks for the extra test coverage too!

@joerunde joerunde merged commit 748cc38 into main Sep 25, 2025
20 checks passed
@joerunde joerunde deleted the fix_rerankers branch September 25, 2025 16:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants