[Feature Request] Prompt lookup decoding

[matichon-vultureprime]

Hi TRT-LLM team,

I see that this technique could be beneficial for Summarization, Context-QA, and Multi-turn chat.

This technique is similar to speculative decoding, but instead of using a draft model, they switch to 'string' matching in the prompt to generate candidate token sequences.

Github : Prompt-lookup-decoding

[Columpio]

@matichon-vultureprime It is already implemented: https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/prompt_lookup/README.md