Added changes for incorporating /rerank support through Huggingface TEI

[ADIthaker]

The code currently runs and passes test, the only thing that needs testing is an actual locally running instance of TEI, to get results from.

Title

Implementing /rerank function

Relevant issues

Fixes #8372

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on (make test-unit)[https://docs.litellm.ai/docs/extras/contributing_code]
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🆕 New Feature

Changes

1. Creates a new HuggingFaceRerank class that has a rerank function that returns a RerankResult as output from function.
2. A test for the same that checks everything except calling the API
3. The format for request and response are set using the reference: https://huggingface.github.io/text-embeddings-inference/#/Text%20Embeddings%20Inference/rerank

Challenges:

1. Currently, due an error within the cargo unit rand_distr, half used by candle, its a bit difficult to setup a TEI instance to test this code against for API calls.
2. The patch for these dependancies has been merged 2 days ago: the trait bound half::bf16: SampleUniform is not satisfied huggingface/candle#2805, so assuming it will take time to propagate, this is the current state of testing things.
3. my make unit-test didnt run, due to lack of bedrock credentials, which shoudln't have been an issue, but not sure what caused it as I wasnt working with it.
4. I made the mistake of not editing the CRLF settings before commiting, and to avoid it getting dirty, Finding a fix for this
   but in the meanwhile here are the files that have major changes
   litellm/llms/huggingface/rerank/handler.py
litellm/llms/huggingface/rerank/transformations.py
tests/litellm/rerank_api/test_rerank_hf.py
litellm/rerank_api/main.py

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Mar 17, 2025 0:57am