Unable to install TensorRT-LLM #2597
Description
Device Details:
Distributor ID: Ubuntu
Description: Ubuntu 20.04.6 LTS
Release: 20.04
Codename: focal
I am trying to build tensorRT-LLM for whisper, and I have followed the steps as mentioned in https://github.com/NVIDIA/TensorRT-LLM/tree/main/examples/whisper#readme, but I am getting the following errors,
When I am using python version 3.8 and pip install -r requirements.txt command, it is giving me the following error
Collecting tensorrt_llm==0.17.0.dev2024121700 (from -r requirements.txt (line 1))
Using cached tensorrt_llm-0.17.0.dev2024121700.tar.gz (1.4 kB)
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing metadata (pyproject.toml) ... error
error: subprocess-exited-with-error
× Preparing metadata (pyproject.toml) did not run successfully.
│ exit code: 1
╰─> [54 lines of output]
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp310-cp310-linux_x86_64.whl against tag cp310-cp310-linux_x86_64
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp312-cp312-linux_aarch64.whl against tag cp312-cp312-linux_aarch64
INFO:wheel-stub:Testing wheel tensorrt_llm-0.17.0.dev2024121700-cp312-cp312-linux_x86_64.whl against tag cp312-cp312-linux_x86_64
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
return download_manual(wheel_directory, distribution, version, config)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 173, in download_manual
raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
Traceback (most recent call last):
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 235, in download_wheel
return download_manual(wheel_directory, distribution, version, config)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 173, in download_manual
raise RuntimeError(f"Didn't find wheel for {distribution} {version}")
RuntimeError: Didn't find wheel for tensorrt-llm 0.17.0.dev2024121700
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
main()
File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
json_out['return_val'] = hook(**hook_input['kwargs'])
File "/home/mcw/anaconda3/lib/python3.8/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 152, in prepare_metadata_for_build_wheel
whl_basename = backend.build_wheel(metadata_directory, config_settings)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/buildapi.py", line 29, in build_wheel
return download_wheel(pathlib.Path(wheel_directory), config_settings)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/wheel.py", line 237, in download_wheel
report_install_failure(distribution, version, config, exception_context)
File "/mnt/mydisk/tmptiny/pip-build-env-q9acj0ez/overlay/lib/python3.8/site-packages/wheel_stub/error.py", line 67, in report_install_failure
raise InstallFailedError(
wheel_stub.error.InstallFailedError:
*******************************************************************************
The installation of tensorrt-llm for version 0.17.0.dev2024121700 failed.
This is a special placeholder package which downloads a real wheel package
from https://pypi.nvidia.com/. If https://pypi.nvidia.com/ is not reachable, we
cannot download the real wheel file to install.
You might try installing this package via
```
$ pip install --extra-index-url https://pypi.nvidia.com/ tensorrt-llm
```
Here is some debug information about your platform to include in any bug
report:
Python Version: CPython 3.8.3
Operating System: Linux 5.15.0-125-generic
CPU Architecture: x86_64
Driver Version: 560.28
CUDA Version: 12.6
*******************************************************************************
[end of output]
note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
However, when I change the python version from 3.8 to 3.10, and run the same command
/home/mcw/anaconda3/envs/py310/compiler_compat/ld: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so: undefined reference to `opal_list_t_class'
collect2: error: ld returned 1 exit status
failure.
removing: _configtest.c _configtest.o
error: Cannot link MPI programs. Check your configuration!!!
ERROR: Failed building wheel for mpi4py
My understanding is that
Problem:
mpi4py only works with python version 3.8 or else it will get error
tensorrt-llm only works with python version 3.10