🐛 Fix fp8 model name check with quantization check

[gkumbhat]

Description

• Replace FP8 model name check with quantization check since user could choose to change their model name and then things can go into weird state, and if they are trying to use pre-compiled model, then it will go into re-compilation

Related Issues

[github-actions]

👋 Hi! Thank you for contributing to vLLM support on Spyre.
Just a reminder: Make sure that your code passes all the linting checks, otherwise your PR won't be able to be merged. To do so, first install the linting requirements, then run format.sh and commit the changes. This can be done with uv directly:

uv sync --frozen --group lint --active --inexact

Or this can be done with pip:

uv pip compile --group lint > requirements-lint.txt
pip install -r requirements-lint.txt
bash format.sh

Now you are good to go 🚀

[prashantgupta24]

lgtm

[ckadner]

What about other quantizations like 4 bit or 8 bit int?

There is some more code from @joerunde and some I recently wrote to do this kind of check, which could be adapted and used here:

• vllm-spyre/vllm_spyre/platform.py

  Line 597 in de0e3d2

  def is_granite_3_8b(cls, model_config: ModelConfig):

• vllm-spyre/vllm_spyre/config/runtime_config_validator.py

  Line 139 in de0e3d2

  def find_known_models_by_model_config(model_config: ModelConfig) -> list[str]:

[joerunde]

We'll have to refactor all these checks once we support more quantization methods, today we only support fp8. I wouldn't mind a refactor to at least pull all of these fp8 checks into one helper instance method in the model class, but we don't need to block this fix on it

I do think this is a separate problem than figuring out which model we're serving though, because any fp8 model has to be handled separately here, not just granite specifically.

[ckadner]

Created a follow-up issue: #537

[joerunde]

Lgtm too

[joerunde]

bot:test
MARKERS="spyre and quantized and not multi"

[joerunde]

I don't believe in the bot tests any more but we might as well try to kick one off to make sure nothing barfs in a new and unusual way