Hello, recently I have been deploying the YOLO11S model on Jetson using TensorRT in your step, but I met some trouble. I dont know why my small model take so much GPU! This has been bothering me for a long time. If you understand, please do offer your advice. Thank you very much.
The GPU usage of the model is 428M and the memory usage is 157M. It was found that the GPU usage rate was extremely high. The maximum performance mode was enabled, and the GPU reached a frequency of 756 MHz, and it was mostly at 98% (as this is an instantaneous observation value, it may not be entirely accurate).
I wonder why my model is so small, but the GPU utilization rate is so high?
Platform Infor strong text mation:
Jetson Orin NX 8G
Jetpack 5.1.5
TensorRT 8.5.2.2
CUDA 11.4
cudnn 8.6
Model Information:
Model: yolo11s
Inference Size: 864*864
Quantization Precision:
FP16
The floating-point operation volume of the model corresponding to ONNX is 21.3 GFLOPs.
Workflow information:
Camera frame rate: 30
Therefore, the computing capacity of this model per second is:
21.3 * 10^9 * 30 / 10^12 = 0.639 TFLOPS
May I ask if this calculation method is correct?
The Jetson Orin NX 8G claims that its INT8 sparse computing power reaches 70 TOPS, and the corresponding FP16 dense computing power can be estimated at 17.5 TFLOPS. My model only occupied 3.65% of its TOPS during operation.
Why is the usage rate so high? How to optimize?
Hello, recently I have been deploying the YOLO11S model on Jetson using TensorRT in your step, but I met some trouble. I dont know why my small model take so much GPU! This has been bothering me for a long time. If you understand, please do offer your advice. Thank you very much.
The GPU usage of the model is 428M and the memory usage is 157M. It was found that the GPU usage rate was extremely high. The maximum performance mode was enabled, and the GPU reached a frequency of 756 MHz, and it was mostly at 98% (as this is an instantaneous observation value, it may not be entirely accurate).
I wonder why my model is so small, but the GPU utilization rate is so high?
Platform Infor strong text mation:
Jetson Orin NX 8G
Jetpack 5.1.5
TensorRT 8.5.2.2
CUDA 11.4
cudnn 8.6
Model Information:
Model: yolo11s
Inference Size: 864*864
Quantization Precision:
FP16
The floating-point operation volume of the model corresponding to ONNX is 21.3 GFLOPs.
Workflow information:
Camera frame rate: 30
Therefore, the computing capacity of this model per second is:
21.3 * 10^9 * 30 / 10^12 = 0.639 TFLOPSMay I ask if this calculation method is correct?
The Jetson Orin NX 8G claims that its INT8 sparse computing power reaches 70 TOPS, and the corresponding FP16 dense computing power can be estimated at 17.5 TFLOPS. My model only occupied 3.65% of its TOPS during operation.
Why is the usage rate so high? How to optimize?