how to compile deepseekv3 ? #2711
Description
I try to build deepseekv3,
command: python3 ./scripts/build_wheel.py
but met the following problems:
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
-- The CXX compiler identification is GNU 11.4.0
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- NVTX is disabled
-- Importing batch manager
-- Importing executor
-- Importing nvrtc wrapper
-- Importing internal cutlass kernels
-- Building PyTorch
-- Building Google tests
-- Building benchmarks
-- Not building C++ micro benchmarks
-- TensorRT-LLM version: 0.17.0.dev2024121700
-- Looking for a CUDA compiler
-- Looking for a CUDA compiler - /usr/local/cuda/bin/nvcc
-- CUDA compiler: /usr/local/cuda/bin/nvcc
-- GPU architectures: 80-real
-- The C compiler identification is GNU 11.4.0
-- The CUDA compiler identification is NVIDIA 12.4.131
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CUDA compiler ABI info
-- Detecting CUDA compiler ABI info - done
-- Check for working CUDA compiler: /usr/local/cuda/bin/nvcc - skipped
-- Detecting CUDA compile features
-- Detecting CUDA compile features - done
-- Found CUDAToolkit: /usr/local/cuda/targets/x86_64-linux/include (found version "12.4.131")
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- CUDA library status:
-- version: 12.4.131
-- libraries: /usr/local/cuda/lib64
-- include path: /usr/local/cuda/targets/x86_64-linux/include
-- pybind11 v3.0.0 dev1
-- Found PythonInterp: /usr/bin/python3 (found suitable version "3.10.12", minimum required is "3.8")
-- Found PythonLibs: /usr/lib/x86_64-linux-gnu/libpython3.10.so
-- Performing Test HAS_FLTO
-- Performing Test HAS_FLTO - Success
-- ========================= Importing and creating target nvinfer ==========================
-- Looking for library nvinfer
-- Library that was found nvinfer_LIB_PATH-NOTFOUND
-- ==========================================================================================
-- CUDAToolkit_VERSION 12.4 is greater or equal than 11.0, enable -DENABLE_BF16 flag
-- CUDAToolkit_VERSION 12.4 is greater or equal than 11.8, enable -DENABLE_FP8 flag
-- Found MPI_C: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi.so (found version "3.1")
-- Found MPI_CXX: /usr/lib/x86_64-linux-gnu/openmpi/lib/libmpi_cxx.so (found version "3.1")
-- Found MPI: TRUE (found version "3.1")
-- COMMON_HEADER_DIRS: /ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/cpp
-- Found Python3: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter Development Development.Module Development.Embed
-- USE_CXX11_ABI is set by python Torch to 0
-- TORCH_CUDA_ARCH_LIST: 8.0
-- Found Python executable at /usr/bin/python3.10
-- Found Python libraries at /usr/lib/x86_64-linux-gnu
-- Found CUDA: /usr/local/cuda (found version "12.4")
-- Found CUDAToolkit: /usr/local/cuda/include (found version "12.4.131")
-- Caffe2: CUDA detected: 12.4
-- Caffe2: CUDA nvcc is: /usr/local/cuda/bin/nvcc
-- Caffe2: CUDA toolkit directory: /usr/local/cuda
-- Caffe2: Header version is: 12.4
-- Found Python: /usr/bin/python3.10 (found version "3.10.12") found components: Interpreter
CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/cuda.cmake:140 (message):
Failed to compute shorthash for libnvrtc.so
Call Stack (most recent call first):
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
CMakeLists.txt:519 (find_package)
CMake Warning (dev) at /opt/cmake/share/cmake-3.28/Modules/FindPackageHandleStandardArgs.cmake:438 (message):
The package name passed to find_package_handle_standard_args (nvtx3) does
not match the name of the calling package (Caffe2). This can lead to
problems in calling code that expects find_package result variables
(e.g., _FOUND) to follow a certain pattern.
Call Stack (most recent call first):
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/cuda.cmake:174 (find_package_handle_standard_args)
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
CMakeLists.txt:519 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.
-- Could NOT find nvtx3 (missing: nvtx3_dir)
CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/public/cuda.cmake:180 (message):
Cannot find NVTX3, find old NVTX instead
Call Stack (most recent call first):
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:86 (include)
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
CMakeLists.txt:519 (find_package)
-- USE_CUDNN is set to 0. Compiling without cuDNN support
-- USE_CUSPARSELT is set to 0. Compiling without cuSPARSELt support
-- USE_CUDSS is set to 0. Compiling without cuDSS support
-- USE_CUFILE is set to 0. Compiling without cuFile support
-- Added CUDA NVCC flags for: -gencode;arch=compute_80,code=sm_80
CMake Warning at /usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:22 (message):
static library kineto_LIBRARY-NOTFOUND not found.
Call Stack (most recent call first):
/usr/local/lib/python3.10/dist-packages/torch/share/cmake/Torch/TorchConfig.cmake:120 (append_torchlib_if_found)
CMakeLists.txt:519 (find_package)
-- Found Torch: /usr/local/lib/python3.10/dist-packages/torch/lib/libtorch.so
-- TORCH_CXX_FLAGS: -D_GLIBCXX_USE_CXX11_ABI=0
'/ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/build.sh' 'libucxx' '-n' '--cmake-args="-DBUILD_SHARED_LIBS=OFF' '-DCMAKE_CXX_FLAGS=-D_GLIBCXX_USE_CXX11_ABI=0"'
-- Configuring done (0.1s)
-- Generating done (0.0s)
-- Build files have been written to: /ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/cpp/build
[1/2] Building CXX object CMakeFiles/ucxx.dir/src/request_am.cpp.o
FAILED: CMakeFiles/ucxx.dir/src/request_am.cpp.o
/usr/bin/c++ -I/ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/cpp/include -I/ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/cpp/src -D_GLIBCXX_USE_CXX11_ABI=0 -O3 -DNDEBUG -std=gnu++17 -fPIC -MD -MT CMakeFiles/ucxx.dir/src/request_am.cpp.o -MF CMakeFiles/ucxx.dir/src/request_am.cpp.o.d -o CMakeFiles/ucxx.dir/src/request_am.cpp.o -c /ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/cpp/src/request_am.cpp
/ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/cpp/src/request_am.cpp: In lambda function:
/ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/cpp/src/request_am.cpp:353:72: error: ‘UCP_AM_SEND_FLAG_COPY_HEADER’ was not declared in this scope; did you mean ‘UCP_AM_SEND_FLAG_EAGER’?
353 | .flags = UCP_AM_SEND_FLAG_REPLY | UCP_AM_SEND_FLAG_COPY_HEADER,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
| UCP_AM_SEND_FLAG_EAGER
ninja: build stopped: subcommand failed.
-- Found ucxx: /ssd2/shijinxiang/tensorrt_llm_test/TensorRT-LLM/3rdparty/ucxx/cpp/build/ucxx-config.cmake (found version "0.41.0")
CMake Error at CMakeLists.txt:569 (file):
file STRINGS file "/usr/local/tensorrt/include/NvInferVersion.h" cannot be
read.
CMake Error at CMakeLists.txt:572 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
CMake Error at CMakeLists.txt:574 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
CMake Error at CMakeLists.txt:572 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
CMake Error at CMakeLists.txt:574 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
CMake Error at CMakeLists.txt:572 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
CMake Error at CMakeLists.txt:574 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
CMake Error at CMakeLists.txt:572 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
CMake Error at CMakeLists.txt:574 (string):
string sub-command REGEX, mode MATCH needs at least 5 arguments total to
command.
-- Building for TensorRT version: .., library version:
CMake Error at CMakeLists.txt:588 (if):
if given arguments:
"LESS" "10"
Unknown arguments specified
-- Configuring incomplete, errors occurred!