Skip to content

Commit 120cd73

Browse files
committed
address lint code issues
Signed-off-by: Sophie du Couédic <[email protected]>
1 parent 2d1de90 commit 120cd73

File tree

1 file changed

+5
-6
lines changed

1 file changed

+5
-6
lines changed

docs/user_guide/cb_testing_in_vllm.md

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -11,10 +11,10 @@ Date: 10th July 2025
1111

1212
### Description
1313
* Runs inference on a set of prompts with continuous batching enabled (number of prompts is parametrizable)
14-
* Prints the generated text for each sequence.
14+
* Prints the generated text for each sequence.
1515
* All the requested sequences are defined in the beginning, there is no requests joining the waiting queue while the decoding of some other request has already started.
1616
* The exact sequence of prefill and decode steps depends on the parameter values `max_num_seqs`, `num-prompts`, `max-tokens`.
17-
* If `--compare-with-CPU` is set, then the output text is compared to the one of hugging face, running on CPU. Note that here the logprobs are not compared, only tokens.
17+
* If `--compare-with-CPU` is set, then the output text is compared to the one of hugging face, running on CPU. Note that here the logprobs are not compared, only tokens.
1818

1919
### Parametrization
2020
* `--model`: the model
@@ -27,7 +27,7 @@ Date: 10th July 2025
2727

2828
## CB tests through unit tests
2929

30-
* **File path (tests targetting CB specifically):** `vllm-spyre/tests/e2e/test_spyre_cb.py`
30+
* **File path (tests targeting CB specifically):** `vllm-spyre/tests/e2e/test_spyre_cb.py`
3131
* **Purpose:** Automated execution to verify that a specific behaviour acts as expected (passing/failing)
3232
* **Usage (when running locally):** `python -m pytest -sv -m "spyre and cb" --forked tests`
3333
* `-s` option: show all the print statements in the code
@@ -37,12 +37,12 @@ Date: 10th July 2025
3737

3838
### Description
3939

40-
Unit tests are designed for automated and systematic execution to verify that CB behaves as expected for different scenarios. For each scenario (i.e. configuration of parameters), the test either passes or fails. When a test suite fails, identifying which specific test case failed is often more informative than the failure message itself. Below is a brief description of the different unit tests targetting CB. The description can also be found in the docstring of the different test functions:
40+
Unit tests are designed for automated and systematic execution to verify that CB behaves as expected for different scenarios. For each scenario (i.e. configuration of parameters), the test either passes or fails. When a test suite fails, identifying which specific test case failed is often more informative than the failure message itself. Below is a brief description of the different unit tests targeting CB. The description can also be found in the docstring of the different test functions:
4141

4242
> All the applicable unit tests in vLLM will eventually also execute with CB enabled in addition to SB, but two test functions specifically target continuous batching correctness: `test_cb_output` and `test_scheduler_cb_steps_tkv`. The other functions found in that files are mostly helper methods, or functions that test CB in aspects more specific to vLLM (such as scheduling constraints). Still it can be interesting to have a look in the code, but their description is skipped here.
4343
4444
#### `test_cb_output`
45-
`test_cb_output` checks the correctness of the output of CB on a set of prompts (4 hardcoded prompts for that test). The output from vllm is compared to this of Hugging Face on CPU.
45+
`test_cb_output` checks the correctness of the output of CB on a set of prompts (4 hardcoded prompts for that test). The output from vllm is compared to this of Hugging Face on CPU.
4646

4747
* **The test passes if:** the logprobs of HF on CPU and vLLM (on Spyre or CPU depending on the backend) are compared, and the test passes only if the pairwise relative differences of the values are all below a threshold: `math.isclose(hf_logprob, vllm_logprob, rel_tol=0.35)`. Otherwise it fails.
4848
> The above applies for sendnn backend, on CPU the tokens need to additionally be exactly the same for the test to pass
@@ -79,4 +79,3 @@ Checking the final output correctness alone is not enough to ensure that CB is c
7979
* `get_params_test_blocks_borders_aligned_prompts`: parametrization for the situation where the prompts are by chance already aligned with the blocks boundaries (no **right** padding required)
8080
* `get_params_test_blocks_borders_misaligned_prompts`: parametrization for the situation where the prompts are misaligned with the block boundaries, and thus **right** padding is required
8181
* ... additional special cases
82-

0 commit comments

Comments
 (0)