Skip to content

Conversation

joerunde
Copy link
Collaborator

@joerunde joerunde commented Oct 14, 2025

Description

There seems to be a small bug where passing a valid env var override will not set the correct value in the environment, only the cache. This could be an issue where the env var needs to be set to be propagated across a fork, but currently we only use it for updating the batching mode for poolers so this shouldn't be a big issue atm.

Edit: forking would use the cache anyway so this shouldn't be a bug. But the logic for handling the environment during overrides was a bit funky- this PR simplifies that code and adds explicit tests covering the expected behavior.

This PR fixes that, adding some tests to cover our envs package. Since there was an expensive e2e test covering the sendnn_decoder backwards compatibility, I moved it to this new test file so that we only have to test the config override instead of loading and running a whole model.

Copy link

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

Signed-off-by: Joe Runde <[email protected]>
@waleedqk
Copy link
Collaborator

bot:test
TEST_FILE=tests/e2e/test_spyre_basic.py MARKERS="spyre"

@joerunde joerunde changed the title 🐛 Fixup env var overrides simplify test for sendnn_decoder backwards compatibility ♻️ Simplify env var overrides and add tests Oct 14, 2025
Copy link
Collaborator

@tjohnson31415 tjohnson31415 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

envs.override("VLLM_SPYRE_NOT_A_CONFIG", "nothing")


def test_sendnn_decoder_backwards_compat(monkeypatch):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🌶️ 🌶️ Test simplification

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants