Skip to content

Commit e6b4b6e

Browse files
committed
add logging
Signed-off-by: Yannick Schnider <[email protected]>
1 parent 4082707 commit e6b4b6e

File tree

1 file changed

+2
-0
lines changed

1 file changed

+2
-0
lines changed

vllm_spyre/v1/worker/spyre_model_runner.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -848,6 +848,8 @@ def _get_num_blocks_available(self) -> int:
848848
# hard coded value for tensor parallel size 4 with the below model
849849
# https://huggingface.co/ibm-granite/granite-3.3-8b-instruct
850850
NUM_BLOCKS_SPYRE = 2080
851+
logger.info("Model granite-3.3-8b-instruct and tensor parallel " \
852+
"size 4 detected. Using NUM_BLOCKS_SPYRE = %d", 2080)
851853
else:
852854
# default value for any other model/ tensor parallel size
853855
NUM_BLOCKS_SPYRE = max_batch_size * min_req_num_blocks

0 commit comments

Comments
 (0)