Description
System Info
- CPU architecture-x86_64
- CPU memory: 123 GB
- GPU Properties: Nvidia 4090, Gpu memory size 24 GB
- Linux version 4090
- libraries (tensorrt llm main branch)
- tensort version: 10.9.0.34
Who can help?
@pathorn @ttim @WilliamTambellini @Superjomn
Information
- The official example scripts
- My own modified scripts
Tasks
- An officially supported task in the
examplesfolder (such as GLUE/SQuAD, ...) - My own task or dataset (give details below)
Reproduction
Steps to reproduce this:
- Build the library from the source.
- start the docker container.
- Try building with the help of wheel file still error occurs.
Expected behavior
Docker container should run
actual behavior
llamaenv) user@CT-DESKTOP:~/TensorRT-LLM$ make -C docker release_run make: Entering directory '/home/user/TensorRT-LLM/docker' docker run --rm -it --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 \ --gpus=all \ --volume /home/user/TensorRT-LLM:/code/tensorrt_llm \ --env "CCACHE_DIR=/code/tensorrt_llm/cpp/.ccache" \ --env "CCACHE_BASEDIR=/code/tensorrt_llm" \ --workdir /app/tensorrt_llm \ --hostname CT-DESKTOP-release \ --name tensorrt_llm-release-user \ --tmpfs /tmp:exec \ tensorrt_llm/release:latest docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error running prestart hook #0: exit status 1, stdout: , stderr: Auto-detected mode as 'legacy' nvidia-container-cli: initialization error: load library failed: libnvidia-ml.so.1: cannot open shared object file: no such file or directory: unknown Run 'docker run --help' for more information make: *** [Makefile:124: release_run] Error 127 make: Leaving directory '/home/user/TensorRT-LLM/docker'
additional notes
Installation using wheel file is also not working.