[EPIC] Support sglang in the inference scheduler

[ezrasilvera]

Integrate sglang (as additional inference engine to vLLM)

• Investigate the dependency of the inference-scheduler on vLLM specifics
• Support Prefill/Decode Disaggregation Add SGLang Connector for Prefill/Decode Disaggregation llm-d-routing-sidecar#64
• ...

[nirrozenbaum]

you should probably start with https://github.com/kubernetes-sigs/gateway-api-inference-extension
more specifically, there is some issue about supporting SgLang here:
kubernetes-sigs/gateway-api-inference-extension#1141

we should have (at least) some initial support.

[ezrasilvera]

Thanks @nirrozenbaum . Sounds reasonable but maybe we should add this in llm-d/llm-d#403 and in parallel find the specific needed integration changes in the scheduler level. I updated it to include your reference, hope it's OK.