You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
*`-m "spyre and cb"`: runs the tests with configurations marked as "spyre" and "cb" only
70
73
71
74
!!! tip
72
-
To run a test with a different model than the default `ibm-ai-platform/micro-g3.3-8b-instruct-1b`, you can run the test with `VLLM_SPYRE_TEST_MODEL_LIST` environment variable set to the targer model, for example:
75
+
To run a test with a different model than the default `ibm-ai-platform/micro-g3.3-8b-instruct-1b`, you can run the test with `VLLM_SPYRE_TEST_MODEL_LIST` environment variable set to the targer model, for example:
@@ -97,7 +100,6 @@ Output tests checks the correctness of the output of CB on a set of prompts. For
97
100
This applies for sendnn backend, on CPU the tokens need to additionally be exactly the same for the test to pass
98
101
* The test passes if: the logprobs of HF on CPU and vLLM (on Spyre or CPU depending on the backend) are compared, and the test passes only if the pairwise relative differences of the values are all below a threshold: `math.isclose(hf_logprob, vllm_logprob, rel_tol=0.35)`. Otherwise it fails. There is no logic that takes into account the fact that the tokens might becomes different at some point, making the logits diverging.
99
102
100
-
101
103
#### Scheduler Steps Tests
102
104
103
105
See [Scheduler Steps Tests](tests/scheduler_steps_tests.md)
0 commit comments