🐛 fix a bug in tests, add DISABLE_ASSERTS #375

prashantgupta24 · 2025-08-12T23:17:27Z

Description

Added DISABLE_ASSERTS to the scheduling tests - It's very useful if you want to debug without actually asserting the values
Fixed a bug where server_args.extend should have be called in tests

github-actions · 2025-08-12T23:17:35Z

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

joerunde · 2025-08-13T15:36:31Z

tests/e2e/test_spyre_online.py

 from spyre_util import get_spyre_backend_list, get_spyre_model_list


+def _check_result(client, model, max_tokens=8, temperature=0.0, n=1) -> None:


tests/e2e/test_spyre_online.py

prashantgupta24 · 2025-08-13T21:56:41Z

bot:test
TEST_FILE=tests/e2e/test_spyre_online.py MARKERS="spyre"

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 · 2025-08-13T22:00:18Z

bot:test
TEST_FILE=tests/e2e/test_spyre_online.py MARKERS="spyre"

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 · 2025-08-14T20:40:57Z

tests/conftest.py

            "VLLM_SPYRE_DYNAMO_BACKEND": backend
        }
-        server_args = [
+        server_args.extend([


found a 🐛 !

Signed-off-by: Prashant Gupta <[email protected]>

yannicks1 · 2025-08-15T14:54:15Z

just seen this... the modification to the model runner can be removed (see #384). The two internal issues on the tracker were coupled I guess. does this solve the remaining part of (internal) issue 977?

Signed-off-by: Prashant Gupta <[email protected]>

tests/scheduling_utils.py

with vllm_spyre:0.7.0 ``` tests/e2e/test_spyre_basic.py::test_output[max_num_seqs(4)-sendnn-TP(4)-ibm-granite/granite-3.3-8b-instruct-cb ``` fails with ``` (VllmWorker rank=1 pid=62989) ERROR 08-15 17:09:36 [multiproc_executor.py:522] File "/usr/local/lib/python3.12/site-packages/torch_sendnn/backends.py", line 2440, in update_graph (VllmWorker rank=1 pid=62989) ERROR 08-15 17:09:36 [multiproc_executor.py:522] self.meta["gl_op"] = GraphLoaderOp(self.meta["g2"], self) (VllmWorker rank=1 pid=62989) ERROR 08-15 17:09:36 [multiproc_executor.py:522] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (VllmWorker rank=1 pid=62989) ERROR 08-15 17:09:36 [multiproc_executor.py:522] File "/usr/local/lib/python3.12/site-packages/torch_sendnn/backends.py", line 1819, in __init__ (VllmWorker rank=1 pid=62989) ERROR 08-15 17:09:36 [multiproc_executor.py:522] raise RuntimeError(f"GraphLoader CompileGraph failed: {s}") (VllmWorker rank=1 pid=62989) ERROR 08-15 17:09:36 [multiproc_executor.py:522] RuntimeError: GraphLoader CompileGraph failed: compile_graph failed ``` so we don't need specific tests! Signed-off-by: Prashant Gupta <[email protected]>

Signed-off-by: Prashant Gupta <[email protected]>

tests/scheduling_utils.py

Signed-off-by: Prashant Gupta <[email protected]>

wallashss · 2025-08-15T17:34:12Z

Thanks for rename the title, it is much better! 😄
(I was about to suggest it)

maxdebayser · 2025-08-18T15:00:09Z

Can you add more information in the PR description as to why the DISABLE_ASSERT is needed?

prashantgupta24 · 2025-08-18T15:30:42Z

DISABLE_ASSERT

We already have it in tests/spyre_util.py. It's very useful if you want to debug without actually asserting the values. Let me update the PR description

Signed-off-by: Prashant Gupta <[email protected]>

maxdebayser

LGTM

prashantgupta24 requested review from rafvasq and sducouedic as code owners August 12, 2025 23:17

joerunde reviewed Aug 13, 2025

View reviewed changes

tests/e2e/test_spyre_online.py Outdated Show resolved Hide resolved

🧪 add failing test for batch requests

eeb87f0

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 force-pushed the cb-batch-failing branch from 7406f15 to eeb87f0 Compare August 13, 2025 22:00

🐛 fix bug

867fce5

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 commented Aug 14, 2025

View reviewed changes

🐛 fix failing condition

d689080

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 requested review from nikolaospapandreou, tdoublep and yannicks1 as code owners August 14, 2025 23:39

prashantgupta24 added 2 commits August 14, 2025 16:42

🎨 add DISABLE_ASSERTS

5002ca9

Signed-off-by: Prashant Gupta <[email protected]>

🎨 fmt

3f3bcb6

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 changed the title ~~🧪 add failing test for batch requests~~ fix some stuff? Aug 14, 2025

prashantgupta24 added 5 commits August 15, 2025 08:31

⏪ revert to main

a7776a2

Signed-off-by: Prashant Gupta <[email protected]>

Merge remote-tracking branch 'upstream/main' into cb-batch-failing

1563359

🚧 some scheduler tests

15d1388

Signed-off-by: Prashant Gupta <[email protected]>

🔥 keep only tp 4

91b6b86

Signed-off-by: Prashant Gupta <[email protected]>

🎨 fmt

ab4d822

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 commented Aug 15, 2025

View reviewed changes

tests/scheduling_utils.py Outdated Show resolved Hide resolved

prashantgupta24 added 2 commits August 15, 2025 10:29

⏪ revert DISABLE_ASSERTS to False

f79aee6

Signed-off-by: Prashant Gupta <[email protected]>

wallashss reviewed Aug 15, 2025

View reviewed changes

tests/scheduling_utils.py Outdated Show resolved Hide resolved

🎨 change comment

01fe49d

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 changed the title ~~fix some stuff?~~ 🐛 fix a bugs in tests, add DISABLE_ASSERTS Aug 15, 2025

prashantgupta24 changed the title ~~🐛 fix a bugs in tests, add DISABLE_ASSERTS~~ 🐛 fix a bug in tests, add DISABLE_ASSERTS Aug 15, 2025

Merge branch 'main' into cb-batch-failing

b72f54e

Signed-off-by: Prashant Gupta <[email protected]>

prashantgupta24 enabled auto-merge (squash) August 18, 2025 15:34

github-actions bot added the ready label Aug 18, 2025

maxdebayser approved these changes Aug 18, 2025

View reviewed changes

prashantgupta24 merged commit 46c6ac4 into main Aug 18, 2025
17 of 23 checks passed

prashantgupta24 deleted the cb-batch-failing branch August 18, 2025 16:15

		from spyre_util import get_spyre_backend_list, get_spyre_model_list


		def _check_result(client, model, max_tokens=8, temperature=0.0, n=1) -> None:

🐛 fix a bug in tests, add DISABLE_ASSERTS #375

🐛 fix a bug in tests, add DISABLE_ASSERTS #375

Uh oh!

Conversation

prashantgupta24 commented Aug 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

github-actions bot commented Aug 12, 2025

Uh oh!

joerunde Aug 13, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

prashantgupta24 commented Aug 13, 2025

Uh oh!

prashantgupta24 commented Aug 13, 2025

Uh oh!

prashantgupta24 Aug 14, 2025

Choose a reason for hiding this comment

Uh oh!

yannicks1 commented Aug 15, 2025

Uh oh!

Uh oh!

Uh oh!

wallashss commented Aug 15, 2025

Uh oh!

maxdebayser commented Aug 18, 2025

Uh oh!

prashantgupta24 commented Aug 18, 2025

Uh oh!

maxdebayser left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

prashantgupta24 commented Aug 12, 2025 •

edited

Loading