Speed up NLI Scoring

[dylanbouchard]

Is your feature request related to a problem? Please describe.
Currently, NLI-based scoring is consistently the most effective UQ method for hallucination detection (Semantic Entropy, Semantic Density, Noncontradiction Probability, Entailment Probability, Number of Semantic Sets). NLI models classify a pair of texts as contradiction/entailment/neutral, which is very useful for extracting hallucination likelihood signals from LLM responses sampled from the same prompt. However, these models tend to be very slow and are the primary bottleneck for latency with the associated scorers.

Describe the solution you'd like
We would like to speed up the NLI scoring. A few options to explore:

• Batching for faster inference
• Using the Accelerator class: from accelerate import Accelerator
• Enable multiple GPUs

Describe alternatives you've considered
Status quo

Additional context
Accelerator: https://huggingface.co/docs/accelerate/en/package_reference/accelerator

Relevant code in UQLM: https://github.com/cvs-health/uqlm/blob/main/uqlm/nli/nli.py#L85-L90

[dylanbouchard]

@NamrataWalanj7

[NamrataWalanj7]

Picking up this open issue