An error occurred while initializing the CUDA installation using Conda on DGX Spark. #10
Description
An error occurred while initializing the CUDA installation using Conda on DGX Spark：
(unsloth_studio) root@spark-3b21:/home/userspcae/UnslothStudio# unsloth studio setup
╔══════════════════════════════════════╗
║ Unsloth Studio Setup Script ║
╚══════════════════════════════════════╝
✅ Frontend already built and up to date -- skipping Node/npm check.
finished finding best python
✅ Using python3.12 (3.12.3) — compatible (3.11.x – 3.13.x)
[====================] 11/11 finalizing
✅ Python dependencies installed
Pre-installing transformers 5.x for newer model support...
✅ Transformers 5.x pre-installed to /root/.unsloth/studio/.venv_t5/
Building llama-server for GGUF inference...
Building with CUDA support (nvcc: /usr/bin/nvcc)...
GPU compute capabilities: 121 -- limiting build to detected archs
❌ cmake llama.cpp failed (exit code 1):
-- The C compiler identification is GNU 13.3.0
-- The CXX compiler identification is GNU 13.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE=Release
-- Found Git: /usr/bin/git (found version "2.43.0")
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: aarch64
-- GGML_SYSTEM_ARCH: ARM
-- Including CPU backend
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- ARM detected
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E
-- Performing Test GGML_COMPILER_SUPPORTS_FP16_FORMAT_I3E - Failed
CMake Warning at ggml/src/ggml-cpu/CMakeLists.txt:146 (message):
ARM -march/-mcpu not found, -mcpu=native will be used
Call Stack (most recent call first):
ggml/src/CMakeLists.txt:445 (ggml_add_cpu_backend_variant_impl)
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod
-- Performing Test GGML_MACHINE_SUPPORTS_dotprod - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nodotprod
-- Performing Test GGML_MACHINE_SUPPORTS_nodotprod - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm
-- Performing Test GGML_MACHINE_SUPPORTS_i8mm - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_noi8mm
-- Performing Test GGML_MACHINE_SUPPORTS_noi8mm - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_sve
-- Performing Test GGML_MACHINE_SUPPORTS_sve - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nosve
-- Performing Test GGML_MACHINE_SUPPORTS_nosve - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_sme
-- Performing Test GGML_MACHINE_SUPPORTS_sme - Failed
-- Performing Test GGML_MACHINE_SUPPORTS_nosme
-- Performing Test GGML_MACHINE_SUPPORTS_nosme - Failed
-- Checking for ARM features using flags:
-- -mcpu=native
-- Performing Test HAVE_DOTPROD
-- Performing Test HAVE_DOTPROD - Failed
-- Performing Test HAVE_SVE
-- Performing Test HAVE_SVE - Success
-- Performing Test HAVE_MATMUL_INT8
-- Performing Test HAVE_MATMUL_INT8 - Failed
-- Performing Test HAVE_FMA
-- Performing Test HAVE_FMA - Success
-- Performing Test HAVE_FP16_VECTOR_ARITHMETIC
-- Performing Test HAVE_FP16_VECTOR_ARITHMETIC - Success
-- Performing Test HAVE_SME
-- Performing Test HAVE_SME - Failed
-- Adding CPU backend variant ggml-cpu: -mcpu=native
-- Found CUDAToolkit: /usr/include (found version "12.0.140")
-- CUDA Toolkit found
CMake Error at /usr/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:780 (message):
Compiling the CUDA compiler identification source file
"CMakeCUDACompilerId.cu" failed.
Compiler: /usr/bin/nvcc
Build flags: --threads=0
Id flags: --keep;--keep-dir;tmp -v
The output was:
255
..........
#$ cudafe++ --c++17 --gnu_version=130300 --display_error_number
--orig_src_file_name "CMakeCUDACompilerId.cu" --orig_src_path_name
"/root/.unsloth/llama.cpp/build/CMakeFiles/3.28.3/CompilerIdCUDA/CMakeCUDACompilerId.cu"
--allow_managed --unsigned_chars --unsigned_wchar_t --m64 --parse_templates
--gen_c_file_name "tmp/CMakeCUDACompilerId.cudafe1.cpp" --stub_file_name
"CMakeCUDACompilerId.cudafe1.stub.c" --gen_module_id_file
--module_id_file_name "tmp/CMakeCUDACompilerId.module_id"
"tmp/CMakeCUDACompilerId.cpp4.ii" >
/tmp/tmpxft_00007be0_00000000-3_11f831d0_stdout
2>/tmp/tmpxft_00007be0_00000000-3_11f831d0_stderr
--error 0x2 --
/usr/include/aarch64-linux-gnu/bits/math-vector.h(96): error: identifier
"__Float32x4_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(97): error: identifier
"__Float64x2_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(106): error: identifier
"__SVFloat32_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(107): error: identifier
"__SVFloat64_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(108): error: identifier
"__SVBool_t" is undefined
5 errors detected in the compilation of "CMakeCUDACompilerId.cu".
Compiling the CUDA compiler identification source file
"CMakeCUDACompilerId.cu" failed.
Compiler: /usr/bin/nvcc
Build flags:
Id flags: --keep;--keep-dir;tmp -v
The output was:
2
#$ NVVM_BRANCH=nvvm
#$ SPACE=
#$ CUDART=cudart
#$ HERE=/usr/lib/nvidia-cuda-toolkit/bin
#$ THERE=/usr/lib/nvidia-cuda-toolkit/bin
#$ TARGET_SIZE=
#$ TARGET_DIR=
#$ TARGET_SIZE=64
#$ NVVMIR_LIBRARY_DIR=/usr/lib/nvidia-cuda-toolkit/libdevice
#$
PATH=/usr/lib/nvidia-cuda-toolkit/bin:/root/.local/bin:/root/.unsloth/studio/.venv/bin:/root/miniconda3/envs/unsloth_studio/bin:/root/miniconda3/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:$GOROOT/bin:$GOPATH/bin:$RUSTPATH/bin
#$ LIBRARIES= -L/usr/lib/aarch64-linux-gnu/stubs
-L/usr/lib/aarch64-linux-gnu
#$ rm tmp/a_dlink.reg.c
#$ gcc -D__CUDA_ARCH_LIST__=520 -E -x c++ -D__CUDACC__ -D__NVCC__
-D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=0
-D__CUDACC_VER_BUILD__=140 -D__CUDA_API_VER_MAJOR__=12
-D__CUDA_API_VER_MINOR__=0 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include
"cuda_runtime.h" "CMakeCUDACompilerId.cu" -o
"tmp/CMakeCUDACompilerId.cpp4.ii"
#$ cudafe++ --c++17 --gnu_version=130300 --display_error_number
--orig_src_file_name "CMakeCUDACompilerId.cu" --orig_src_path_name
"/root/.unsloth/llama.cpp/build/CMakeFiles/3.28.3/CompilerIdCUDA/CMakeCUDACompilerId.cu"
--allow_managed --unsigned_chars --unsigned_wchar_t --m64 --parse_templates
--gen_c_file_name "tmp/CMakeCUDACompilerId.cudafe1.cpp" --stub_file_name
"CMakeCUDACompilerId.cudafe1.stub.c" --gen_module_id_file
--module_id_file_name "tmp/CMakeCUDACompilerId.module_id"
"tmp/CMakeCUDACompilerId.cpp4.ii"
/usr/include/aarch64-linux-gnu/bits/math-vector.h(96): error: identifier
"__Float32x4_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(97): error: identifier
"__Float64x2_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(106): error: identifier
"__SVFloat32_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(107): error: identifier
"__SVFloat64_t" is undefined
/usr/include/aarch64-linux-gnu/bits/math-vector.h(108): error: identifier
"__SVBool_t" is undefined
5 errors detected in the compilation of "CMakeCUDACompilerId.cu".
--error 0x2 --
Call Stack (most recent call first):
/usr/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-3.28/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
/usr/share/cmake-3.28/Modules/CMakeDetermineCUDACompiler.cmake:135 (CMAKE_DETERMINE_COMPILER_ID)
ggml/src/ggml-cuda/CMakeLists.txt:58 (enable_language)
-- Configuring incomplete, errors occurred!
...