Add TensorRT-LLM support as another backend #205
Labels
needs-kind
Indicates a PR lacks a label and requires one.
needs-priority
Indicates a PR lacks a label and requires one.
needs-triage
Indicates an issue or PR lacks a label and requires one.
What would you like to be added:
Add support for https://github.com/NVIDIA/TensorRT-LLM?tab=readme-ov-file
see blog like https://developer.nvidia.com/blog/optimizing-inference-on-llms-with-tensorrt-llm-now-publicly-available/
Why is this needed:
Completion requirements:
This enhancement requires the following artifacts:
The artifacts should be linked in subsequent comments.
The text was updated successfully, but these errors were encountered: