GPU CUDA

[Alvin0523]

Here's a concise, professional GitHub issue template with all diagnostics needed to debug your GPU utilization problem. Copy/paste this into the ros2_yolos_cpp repo issues:

---

Issue: GPU utilization <3% despite ONNX Runtime GPU build and ENABLE_GPU=ON

Problem Description

Node logs "Inference device: GPU (CUDA)" after lifecycle activation, but:

• GPU utilization remains <3% (nvidia-smi)
• GPU memory usage stuck at ~288 MiB (only image buffers)
• Inference frequency 5–6 Hz (matches CPU performance)
• Expected: 40–100+ FPS with RTX 5080 + ONNX Runtime GPU

This indicates silent CPU fallback despite GPU archive being used.

Environment

OS: Ubuntu 22.04 (WSL2 on Windows 11)
ROS 2 Distribution: Humble
GPU: NVIDIA GeForce RTX 5080
Driver: 591.59 (Windows host)
CUDA: 12.8 (nvcc release 12.8, V12.8.61)
cuDNN: 9.19.0 (libcudnn9-cuda-12)
ONNX Runtime: 1.20.1 GPU build (downloaded via YOLOs-CPP build.sh 1.20.1 1)
ROS 2 Package: ros2_yolos_cpp (built with -DENABLE_GPU=ON)
Model: yolo11n.onnx (640x640)

Steps to Reproduce

# 1. Build standalone YOLOs-CPP with GPU ONNX Runtime
cd ~/YOLOs-CPP
./build.sh 1.20.1 1  # Confirmed: onnxruntime-linux-x64-gpu-1.20.1 downloaded

# 2. Build ROS 2 wrapper
cd ~/ros2_ws
colcon build --packages-select ros2_yolos_cpp --cmake-args -DENABLE_GPU=ON

# 3. Launch detector + publish images
ros2 launch ros2_yolos_cpp detector.launch.py \
  model_path:=src/ros2_yolos_cpp/models/yolo11n.onnx \
  labels_path:=src/ros2_yolos_cpp/models/coco.names \
  use_gpu:=true \
  image_topic:=/camera/image_raw &


#### Observed Behavior
| Metric | Value | Expected |
|--------|-------|----------|
| `nvidia-smi` utilization | 2–3% | 40–85% |
| `nvidia-smi` memory | 288 MiB | 1.8–3.0 GiB |
| Inference frequency | 5–6 Hz | 40–100+ Hz |
| Node log | `"Inference device: GPU (CUDA)"` | Same (misleading) |
| ONNX Runtime archive | `onnxruntime-linux-x64-gpu-1.20.1.tgz` | ✅ Correct |

#### Critical Diagnostics Performed

✅ **CUDA/cuDNN installed**:
```bash
$ nvcc --version
release 12.8, V12.8.61

$ dpkg -l | grep cudnn
ii  libcudnn9-cuda-12  9.19.0.56-1  amd64