Search results

Errors running sanity check test following installation guide

bugSomething isn't workingSomething isn't working

InstallationSetting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.Setting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.

Investigating

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5495 In NVIDIA/TensorRT-LLM;

· stmcginnis opened on Jun 25, 2025

Compatibility with HF LLaMA Codebase

feature requestNew feature or request. This includes new model, dtype, functionality supportNew feature or request. This includes new model, dtype, functionality support

Generic RuntimeGeneral operational aspects of TRTLLM execution not in other categories.General operational aspects of TRTLLM execution not in other categories.

Investigating

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5492 In NVIDIA/TensorRT-LLM;

· Saeedmatt3r opened on Jun 25, 2025

[Perf] Conditionally enable SWAP AB for speculative decoding

Community want to contributePRs initiated from CommunityPRs initiated from Community

Investigating

PerformanceTRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.TRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5403 In NVIDIA/TensorRT-LLM;

· zoheth opened on Jun 23, 2025

invalid request_id in <code>MTPSpecMetadata</code>

bugSomething isn't workingSomething isn't working

Investigating

Speculative DecodingRelated to MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafterRelated to MTP/Eagle/Medusa/Lookahead/Prompt-Lookup-Decoding/Draft-Target-Model/ReDrafter

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5386 In NVIDIA/TensorRT-LLM;

· k-l-lambda opened on Jun 20, 2025

Mixtral-8x7B-Instruct awq-w4a8 output shows duplicated Chinese text

bugSomething isn't workingSomething isn't working

Investigating

Low PrecisionLower-precision formats (INT8/INT4/FP8) for TRTLLM quantization (AWQ, GPTQ).Lower-precision formats (INT8/INT4/FP8) for TRTLLM quantization (AWQ, GPTQ).

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5379 In NVIDIA/TensorRT-LLM;

· wanzhenchn opened on Jun 20, 2025

Poor performance after FP8 Quantization for Llama 3.1 on PyTorch backend

bugSomething isn't workingSomething isn't working

Investigating

PerformanceTRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.TRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5370 In NVIDIA/TensorRT-LLM;

· geaned opened on Jun 19, 2025

How to run Qwen3 using triton-server + trtllm

Investigating

questionFurther information is requestedFurther information is requested

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Triton BackendRelated to NVIDIA Triton Inference Server backendRelated to NVIDIA Triton Inference Server backend

Status: Open.

#5310 In NVIDIA/TensorRT-LLM;

· ezioliao opened on Jun 18, 2025

Abnormal Performance Scaling of W4AFP8 vs FP8 on H20-141G with Deepseek-R1 Models

Investigating

PerformanceTRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.TRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5127 In NVIDIA/TensorRT-LLM;

· Nekofish-L opened on Jun 11, 2025

guided decoding parameters for tensorrt_llm backend must be present even if not needed

bugSomething isn't workingSomething isn't working

Investigating

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Triton BackendRelated to NVIDIA Triton Inference Server backendRelated to NVIDIA Triton Inference Server backend

Status: Open.

#5099 In NVIDIA/TensorRT-LLM;

· InCogNiTo124 opened on Jun 10, 2025

disaggregated service MPI communicate failed,pls help to check,thanks

bugSomething isn't workingSomething isn't working

Disaggregated ServingDeploying TRTLLM with separated, distributed components (params, kv-cache, compute). Arch & perf.Deploying TRTLLM with separated, distributed components (params, kv-cache, compute). Arch & perf.

Investigating

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#5012 In NVIDIA/TensorRT-LLM;

· w066650 opened on Jun 9, 2025

How is the performance of the model with pytorch as the backend

Investigating

PerformanceTRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.TRTLLM model inference speed, throughput, efficiency. Latency, benchmarks, regressions, opts.

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#4745 In NVIDIA/TensorRT-LLM;

· oppolll opened on May 29, 2025

No module named 'tensorrt_llm.bindings

InstallationSetting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.Setting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.

Investigating

triagedIssue has been triaged by maintainersIssue has been triaged by maintainers

Status: Open.

#4458 In NVIDIA/TensorRT-LLM;

· Shegun93 opened on May 19, 2025