Update on the development branch

[kaiyux]

Hi,

The TensorRT-LLM team is pleased to announce that we are pushing an update to the development branch (and the Triton backend) this June 25, 2024.

This update includes:

• API
  • [BREAKING CHANGE] Remove unnecessary --weight_only_precision argument from trtllm-build command.
• Bug fixes
  • Raise error when autopp detects unsupported quant plugin Unsupported auto parallel + int4 quantization on models #1626.
  • Fix the issue that shared_embedding_table is not being set when loading Gemma [GEMMA] from_hugging_face not setting share_embedding_table to True leading to incapacity to load Gemma #1799, thanks to the contribution from @mfuntowicz.
• Benchmark
  • Add Medusa choices to the gptManagerBenchmark.
• Infra
  • Base Docker image for TensorRT-LLM is updated to nvcr.io/nvidia/pytorch:24.05-py3.
  • Base Docker image for TensorRT-LLM backend is updated to nvcr.io/nvidia/tritonserver:24.05-py3.
  • The dependent TensorRT version is updated to 10.1.

Thanks,
The TensorRT-LLM Engineering Team