🎨 make max_num_seqs 4 for online test

[prashantgupta24]

Description

For model ibm-ai-platform/micro-g3.3-8b-instruct-1b:
max_num_seqs 2 with TP 4 and CB for test_openai_serving seems to be failing:

numValidElems: 139984597168000 larger than attn mask vector size: 256dataformat_src_: IEEE_INT64

Some notes:

Earlier we had only BS 2 tested with:

1. test_openai_serving which tested with SB and TP 1, 2, 4, and
2. test_openai_serving_cb which tested against CB but with no TP.

Both of them separately passed, but TP (4) with CB and BS 2 for test_spyre_online is failing (but TP (2) with CB and BS 2 passes)

Also the full model passes:

tests/e2e/test_spyre_online.py::test_openai_serving[max_model_len(256)-max_num_seqs(2)-cb-sendnn-TP(4)-ibm-granite/granite-3.3-8b-instruct] PASSED

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[joerunde]

F