Skip to content

Unable to install tensorrt_llm on Amazon Linux 2. #2769

New issue
New issue
Open
Open
Unable to install tensorrt_llm on Amazon Linux 2.#2769
Assignees
rishic3
Labels
InstallationSetting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.InvestigatingtriagedIssue has been triaged by maintainerswaiting for feedback
@eduardzl

Description

@eduardzl
eduardzl
opened on Feb 9, 2025

Hello.
Trying to install tensorrt_llm in SageMaker Jupyter Notebook.
The command :
pip install --extra-index-url https://pypi.nvidia.com tensorrt_llm

The installation fails when installing tensorrt_cu12_libs.
Here is the installation log :

Collecting nvidia-ml-py<13.0.0a0,>=12.0.0 (from pynvml>=11.5.0->tensorrt_llm)
....
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [62 lines of output]
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_28_x86_64.whl against tag py3-none-manylinux_2_28_x86_64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_28_x86_64.whl against tag py2-none-manylinux_2_28_x86_64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_31_aarch64.whl against tag py3-none-manylinux_2_31_aarch64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_31_aarch64.whl against tag py2-none-manylinux_2_31_aarch64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-win_amd64.whl against tag py2-none-win_amd64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-win_amd64.whl against tag py3-none-win_amd64
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-3is501it/overlay/lib/python3.12/site-packages/wheel_stub/wheel.py", line 249, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-3is501it/overlay/lib/python3.12/site-packages/wheel_stub/wheel.py", line 185, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      Traceback (most recent call last):
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-3is501it/overlay/lib/python3.12/site-packages/wheel_stub/wheel.py", line 249, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-3is501it/overlay/lib/python3.12/site-packages/wheel_stub/wheel.py", line 185, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      RuntimeError: Didn't find wheel for tensorrt-cu12-libs 10.8.0.43
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/home/ec2-user/SageMaker/envs/conda_film_tensorrt-llm/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
        File "/home/ec2-user/SageMaker/envs/conda_film_tensorrt-llm/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/envs/conda_film_tensorrt-llm/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 178, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-3is501it/overlay/lib/python3.12/site-packages/wheel_stub/buildapi.py", line 29, in build_wheel
          return download_wheel(pathlib.Path(wheel_directory), config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-3is501it/overlay/lib/python3.12/site-packages/wheel_stub/wheel.py", line 251, in download_wheel
          report_install_failure(distribution, version, config, exception_context)
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-3is501it/overlay/lib/python3.12/site-packages/wheel_stub/error.py", line 67, in report_install_failure
          raise InstallFailedError(
      wheel_stub.error.InstallFailedError:
      *******************************************************************************
      
      The installation of tensorrt-cu12-libs for version 10.8.0.43 failed.
      
      This is a special placeholder package which downloads a real wheel package
      from [https://pypi.nvidia.com/.](https://pypi.nvidia.com/) If https://pypi.nvidia.com/ is not reachable, we
      cannot download the real wheel file to install.
      
      You might try installing this package via
     
      $ pip install --extra-index-url https://pypi.nvidia.com/ tensorrt-cu12-libs
      
      
      Here is some debug information about your platform to include in any bug
      report:
      
      Python Version: CPython 3.12.8
      Operating System: Linux 5.10.230-223.885.amzn2.x86_64
      CPU Architecture: x86_64
      Driver Version: 550.144
      CUDA Version: 12.4
      
      *******************************************************************************
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Attemps to install tensorrt-cu12 also fail with same error.
Command : pip install --extra-index-url https://pypi.nvidia.com/ tensorrt-cu12==10.8.0.43
Error :

Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... error
  error: subprocess-exited-with-error
  
  × Preparing metadata (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [62 lines of output]
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_28_x86_64.whl against tag py3-none-manylinux_2_28_x86_64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_28_x86_64.whl against tag py2-none-manylinux_2_28_x86_64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_31_aarch64.whl against tag py2-none-manylinux_2_31_aarch64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-manylinux_2_31_aarch64.whl against tag py3-none-manylinux_2_31_aarch64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-win_amd64.whl against tag py3-none-win_amd64
      INFO:wheel-stub:Testing wheel tensorrt_cu12_libs-10.8.0.43-py2.py3-none-win_amd64.whl against tag py2-none-win_amd64
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-kpsxp9me/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 249, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-kpsxp9me/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 185, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      Traceback (most recent call last):
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-kpsxp9me/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 249, in download_wheel
          return download_manual(wheel_directory, distribution, version, config)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-kpsxp9me/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 185, in download_manual
          raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
      RuntimeError: Didn't find wheel for tensorrt-cu12-libs 10.8.0.43
      
      During handling of the above exception, another exception occurred:
      
      Traceback (most recent call last):
        File "/home/ec2-user/SageMaker/envs/conda_film_tensorrt-llm/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 389, in <module>
          main()
        File "/home/ec2-user/SageMaker/envs/conda_film_tensorrt-llm/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 373, in main
          json_out["return_val"] = hook(**hook_input["kwargs"])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/envs/conda_film_tensorrt-llm/lib/python3.11/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 178, in prepare_metadata_for_build_wheel
          whl_basename = backend.build_wheel(metadata_directory, config_settings)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-kpsxp9me/overlay/lib/python3.11/site-packages/wheel_stub/buildapi.py", line 29, in build_wheel
          return download_wheel(pathlib.Path(wheel_directory), config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-kpsxp9me/overlay/lib/python3.11/site-packages/wheel_stub/wheel.py", line 251, in download_wheel
          report_install_failure(distribution, version, config, exception_context)
        File "/home/ec2-user/SageMaker/tmp/pip-build-env-kpsxp9me/overlay/lib/python3.11/site-packages/wheel_stub/error.py", line 67, in report_install_failure
          raise InstallFailedError(
      wheel_stub.error.InstallFailedError:
      *******************************************************************************
      
      The installation of tensorrt-cu12-libs for version 10.8.0.43 failed.
      
      This is a special placeholder package which downloads a real wheel package
      from [https://pypi.nvidia.com/.](https://pypi.nvidia.com/) If https://pypi.nvidia.com/ is not reachable, we
      cannot download the real wheel file to install.
      
      You might try installing this package via
      
      $ pip install --extra-index-url https://pypi.nvidia.com/ tensorrt-cu12-libs
       
      Here is some debug information about your platform to include in any bug
      report:
      
      Python Version: CPython 3.11.11
      Operating System: Linux 5.10.230-223.885.amzn2.x86_64
      CPU Architecture: x86_64
      Driver Version: 550.144
      CUDA Version: 12.4
      
      *******************************************************************************
      
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

Tried :
pip install --upgrade pip setuptools, didn't help
pip install https://download.pytorch.org/whl/cu124/torch-2.5.1%2Bcu124-cp312-cp312-linux_x86_64.whl - didn't help.

Metadata

Metadata

Assignees

Labels

InstallationSetting up and building TRTLLM: compilation, pip install, dependencies, env config, CMake.InvestigatingtriagedIssue has been triaged by maintainerswaiting for feedback

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions