feat: Add rerank models and rerank API change

[jiayin-nvidia]

What does this PR do?

• Extend the model type to include rerank models.
• Implement rerank() method in inference router.
• Add construct_model_from_identifier to OpenAIMixin to allow customization of model typing/metadata.
• Update documentation.

Test Plan

pytest tests/unit/providers/utils/inference/test_openai_mixin.py

[ehhuang]

do we need this?

Suggested change

	- Rerank models (Experimental): these models reorder the documents based on
	- Rerank models: these models reorder the documents based on

[jiayin-nvidia]

I initially added it since I noticed the rerank API doc is under https://github.com/llamastack/llama-stack/blob/main/docs/static/experimental-llama-stack-spec.html#L1417, but I can definitely remove it if it's not needed.

[mattf]

the changes to apis/inference/inference.py and apis/model/model.py are great

the addition of rerank to the inference router is consistent and useful

the changes to openai_mixin are awkward because openai does not provide /rerank and nvidia is our only rerank impl. i understand the motivation to use as much standard code as possible in the nvidia adapter.

instead of introducing the concept of rerank into openai, what about giving the sub-class an opportunity to construct the model. lift the model construction code out of list_models so sub-classes can override it, e.g.

def construct_model_from_identifier(identifier: str) -> Model:
   if metadata := self.embedding_model_metadata...
   else:
      model = Model(...model_type=ModelType.llm)

then the nvidia provider can impl construct_model_from_identifier with something like -

def construct_model_from_identifier(identifier: str) -> Model:
   if identifier in self.rerank_model_list:
      return Model(...ModelType.rerank)
   return super().construct_model_from_identifier(identifier)

[jiayin-nvidia]

the changes to apis/inference/inference.py and apis/model/model.py are great

the addition of rerank to the inference router is consistent and useful

the changes to openai_mixin are awkward because openai does not provide /rerank and nvidia is our only rerank impl. i understand the motivation to use as much standard code as possible in the nvidia adapter.

instead of introducing the concept of rerank into openai, what about giving the sub-class an opportunity to construct the model. lift the model construction code out of list_models so sub-classes can override it, e.g.

def construct_model_from_identifier(identifier: str) -> Model:
   if metadata := self.embedding_model_metadata...
   else:
      model = Model(...model_type=ModelType.llm)

then the nvidia provider can impl construct_model_from_identifier with something like -

def construct_model_from_identifier(identifier: str) -> Model:
   if identifier in self.rerank_model_list:
      return Model(...ModelType.rerank)
   return super().construct_model_from_identifier(identifier)

Hi @mattf , thank you for your suggestion, this sounds great. I will also update NVIDIA implementation accordingly in #3329.