No module named 'tensorrt_llm.bindings

[Shegun93]

System info
OS -- Linux
Python version -- 3.10
CUDA -- 12.4

I have tried the following steps to install/set up tensorrt_llm. All steps have been taken through official documentation

1. Installing via pip install
   sudo apt-get -y install libopenmpi-dev && pip3 install --upgrade pip setuptools && pip3 install tensorrt_llm
   The installation was successful with no error message, but when checked for tensorrt_llm throughthe import function, the error below was produced

`Python 3.10.0 | packaged by conda-forge | (default, Nov 20 2021, 02:24:10) [GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

import tensorrt
import tensorrt_llm
Invalid MIT-MAGIC-COOKIE-1 keyTraceback (most recent call last):
File "", line 1, in
File "/home/shegun93/TensorRT-LLM/tensorrt_llm/init.py", line 32, in
import tensorrt_llm.functional as functional
File "/home/shegun93/TensorRT-LLM/tensorrt_llm/functional.py", line 28, in
from . import graph_rewriting as gw
File "/home/shegun93/TensorRT-LLM/tensorrt_llm/graph_rewriting.py", line 11, in
from ._utils import trt_gte
File "/home/shegun93/TensorRT-LLM/tensorrt_llm/_utils.py", line 42, in
from tensorrt_llm.bindings import DataType, GptJsonConfig
ModuleNotFoundError: No module named 'tensorrt_llm.bindings'`

Suffix to say I have tried both conda and Python environments for this installation

2. Building from source
   Successfull followed up to Option 2: Build TensorRT-LLM Step-by-Step in the official documentation (Building from source)
   At the point of building the whl pip install ./build/tensorrt_llm*.whl I couldn't progress past 96%

I have equally tried other methods, including suggestions from online sources and picking from the related issues that have been raised on here. no luck yet

[Lifann]

Met a similar error while compiling from source.

rhel-8
python-3.10
cuda-12.8
cudnn-local-repo-rhel8-9.10.2-1
gcc-10.3

export NCCL_LIBRARIES=/usr/local/nccl/lib
export NCCL_INCLUDE_DIR=/usr/local/nccl/include
export USE_CUDNN=1

cd TensorRT-LLM
python3 ./scripts/build_wheel.py               \
        --trt_root "/usr/src/tensorrt"       \
        --nccl_root "/usr/local/tccl"          \
        --cuda_architectures "90-real"

The error text:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/local/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals) 
  File "/data/transfer/TensorRT-LLM/.venv-3.10/lib/python3.10/site-packages/pybind11_stubgen/__main__.py", line 4, in <module>
    main()
  File "/data/transfer/TensorRT-LLM/.venv-3.10/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 320, in main
    run( 
  File "/data/transfer/TensorRT-LLM/.venv-3.10/lib/python3.10/site-packages/pybind11_stubgen/__init__.py", line 359, in run
    QualifiedName.from_str(module_name), importlib.import_module(module_name)
  File "/usr/local/lib/python3.10/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'bindings'

And the following suggestions did not work

Failed to build pybind11 stubgen: Command '"/data/transfer/TensorRT-LLM/.venv-3.10/bin/python3" -m pybind11_stubgen -o . bindings --exit-code' returned non-zero exit status 1.
-- Using virtual environment at: /data/transfer/TensorRT-LLM/.venv-3.10 (Python 3.10)
-- Ensuring virtualenv version >=20.29.1,<22.0 is installed...
-- Virtual environment already exists.
-- Installing requirements from /data/transfer/TensorRT-LLM/requirements-dev.txt into /data/transfer/TensorRT-LLM/.venv-3.10...
-- Found conan executable at: /data/transfer/TensorRT-LLM/.venv-3.10/bin/conan
CMake Build command: 
cmake --build . --config Release --parallel 32 --target build_wheel_targets

[ccs96307]

I got a similar issue: python3 scripts/build_wheel.py

the error message is:

...
[ 98%] Built target executorWorker                                                                                                                                                            
[ 98%] Built target bindings                                                                                                                                                                  
[100%] Built target th_common                                                                                                                                                                 
[100%] Built target build_wheel_targets                                                        
Requirement already satisfied: pybind11-stubgen in ./.venv-3.12/lib/python3.12/site-packages (2.5.4)                                                                                          
Traceback (most recent call last):                                                                                                                                                            
  File "<frozen runpy>", line 198, in _run_module_as_main                                      
  File "<frozen runpy>", line 88, in _run_code                                                                                                                                                
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/__main__.py", line 4, in <module>                                                                         
    main()                                                                                     
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/__init__.py", line 320, in main
    run(
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/__init__.py", line 358, in run
    module = parser.handle_module(
             ^^^^^^^^^^^^^^^^^^^^^
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/parser/mixins/error_handlers.py", line 45, in handle_module
    return super().handle_module(path, module)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/parser/mixins/fix.py", line 207, in handle_module
    result = super().handle_module(path, module)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/parser/mixins/fix.py", line 223, in handle_module
    result = super().handle_module(path, module)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/parser/mixins/fix.py", line 128, in handle_module
    result = super().handle_module(path, module)

...

  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/parser/mixins/fix.py", line 186, in _is_module                
    return importlib.import_module(str(name)) is not None                                                                                                                                     
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                 
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module                                                                                                                 
    return _bootstrap._gcd_import(name[level:], package, level)                                                                                                                               
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                               
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import                                                                                                                             
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load                                                                                                                          
  File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked                                                                                                                 
  File "<frozen importlib._bootstrap>", line 935, in _load_unlocked                            
  File "<frozen importlib._bootstrap_external>", line 995, in exec_module                                                                                                                     
  File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed                                                                                                                
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/torch/__init__.py", line 2126, in <module>
    from torch import _VF as _VF, functional as functional  # usort: skip                                                                                                                     
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                    
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/torch/functional.py", line 8, in <module>
    import torch.nn.functional as F                                                                                                                                                           
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/torch/nn/__init__.py", line 9, in <module>                                                                                 
    from torch.nn import (                                                                     
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/torch/nn/attention/__init__.py", line 8, in <module>                                                                       
    import torch.backends.cuda                                                                                                                                                                
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/torch/backends/__init__.py", line 60, in <module>
    from torch.backends import (                                                                                                                                                              
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/torch/backends/cuda/__init__.py", line 221, in <module>                                                                    
    "default": torch._C._BlasBackend.Default,                                                  
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^                                                                                                                                                  
AttributeError: type object 'torch._C._BlasBackend' has no attribute 'Default'                                                                                                                
Failed to build pybind11 stubgen: Command '"/code/tensorrt_llm/.venv-3.12/bin/python3" -m pybind11_stubgen -o . bindings --exit-code' returned non-zero exit status 1.

and then I tried to run the command: /code/tensorrt_llm/.venv-3.12/bin/python3 -m pybind11_stubgen -o . bindings --exit-code

I got the same error message:

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/__main__.py", line 4, in <module>
    main()
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/__init__.py", line 320, in main
    run(
  File "/code/tensorrt_llm/.venv-3.12/lib/python3.12/site-packages/pybind11_stubgen/__init__.py", line 359, in run
    QualifiedName.from_str(module_name), importlib.import_module(module_name)
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/importlib/__init__.py", line 90, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1324, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'bindings'

[ccs96307]

I found a strange solution, but I'm not sure all the steps was required.

First, I activated my virtual environment, and check the package in my ipython:

source /code/tensorrt_llm/.venv-3.12/bin/activate

ipython

---

And I ensure I have tensorrt_llm.bindings module:

In [1]: import tensorrt_llm.bindings
2025-06-23 14:28:33,358 - INFO - flashinfer.jit: Prebuilt kernels not found, using JIT backend
[TensorRT-LLM] TensorRT-LLM version: 0.21.0rc1

In [2]: import tensorrt_llm.bindings

In [3]:                              

Then I edited the /code/tensorrt_llm/scripts/build_wheel.py file, changed the following line:

f"\"{venv_python}\" -m pybind11_stubgen -o . bindings --exit-code"

to

f"\"{venv_python}\" -m pybind11_stubgen -o . tensorrt_llm.bindings --exit-code"

and then re-run, this time I built successfully.