Skip to content

Commit 9511982

Browse files
🐛 fix runtime msg (#244)
# Description Earlier, the error showed up as ``` RuntimeError: Warmup shape [4, {shape['prompt_length']}, {shape['new_tokens']}] results in a maximum sequence length of {max_seq_len} which is longer that what the model supports ({max_model_len}) ``` After this fix, it shows up correctly: ``` RuntimeError: Warmup shape [4, 128, 20] results in a maximum sequence length of 148 which is longer that what the model supports (128) ``` Signed-off-by: Prashant Gupta <[email protected]>
1 parent c00da14 commit 9511982

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

vllm_spyre/platform.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -214,10 +214,10 @@ def get_warmup_shapes(
214214
if max_seq_len > max_model_len:
215215
raise RuntimeError(
216216
f"Warmup shape [{shape['batch_size']},"
217-
" {shape['prompt_length']}, {shape['new_tokens']}]"
218-
" results in a maximum sequence length of "
219-
"{max_seq_len} which is longer that what the model "
220-
"supports ({max_model_len})")
217+
f" {shape['prompt_length']}, {shape['new_tokens']}]"
218+
f" results in a maximum sequence length of "
219+
f"{max_seq_len} which is longer that what the model "
220+
f"supports ({max_model_len})")
221221
return cls._warmup_shapes
222222

223223
@classmethod

0 commit comments

Comments
 (0)