cv2.cuda_GpuMat()

[SwHaraday]

Search before asking

• I checked the docs, issues, and discussions and could not find an answer.

Question

Hi Glenn,

It's been a while. Thanks to your great work, our yolov5 system has been deployed across 7 lines.

I'm wondering if it is possible that resize and letterbox are done on GPU using cv2.cuda_GpuMat().
I knew these 2 proccesses take 5 to 7 milli-second which is about half of total inference time.
Is it possible to feed cv2.cuda_GpuMat() directly to yolov5s(ver.6.0) model ?

If this is already answered, please kindly guide me.

Thank you for your continuing effort for democratization of object detection.

Additional

No response

[UltralyticsAssistant]

👋 Hello @SwHaraday, thank you for your interest in YOLOv5 🚀 and for sharing that your system is already deployed across 7 lines — that’s awesome to hear 😄
This is an automated response to help get things moving quickly, and an Ultralytics engineer will also assist you soon.
Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.
If this is a 🐛 Bug Report, please provide a minimum reproducible example to help us debug it.
If this is a custom training ❓ Question, please provide as much information as possible, including dataset image examples and training logs, and verify you are following our Tips for Best Training Results.
For performance and preprocessing questions like this one, it would also help if you could share:

• Your exact YOLOv5 version and install method
• Your OpenCV, Python, PyTorch, and CUDA versions
• A small reproducible code snippet showing your current preprocessing and inference pipeline
• Timing breakdowns for resize, letterbox, host-to-device transfer, and inference
• Whether you are using detect.py directly or a custom pipeline

Requirements

Python>=3.8.0 with all requirements.txt installed including PyTorch>=1.8. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on macOS, Windows, and Ubuntu every 24 hours and on every commit.

[pderrenger]

Thanks for the kind words, and credit to the YOLO community as well. YOLOv5 v6.0 does not natively accept cv2.cuda_GpuMat() inputs, since the default detect.py path does resize/letterbox on CPU and the model expects a torch.Tensor; if you want GPU preprocessing, you’ll need a custom wrapper that converts the frame into a CUDA tensor before inference, and it’s also worth checking whether the same bottleneck remains on the latest YOLOv5 code.

[SwHaraday]

Dear pderrenger,

Thank you very much for detailed explanation.
I've confirmed resize/letterbox can be done on GPU.
I started reading class DetectMultiBackend in models/common.py.
I'll find the way to convert cv2.cuda_GpuMat() to torch.Tensor on GPU.

BR Yuji

[pderrenger]

Thanks for the update, Yuji — that’s the right direction: DetectMultiBackend can take a torch.Tensor, so the clean path is usually to keep your GPU preprocessing in a custom loader/wrapper and pass a CUDA BCHW tensor with the expected dtype/normalization into inference; if you run into a specific conversion or shape issue, feel free to share that snippet here and we can take a look.