Yolov5 inference with image size different from 640 on ARTPEC8

[hannaliavoshka]

Is there a way to run the Yolov5 model trained and exported with imgsz = 1280? It doesn't work out of the box, and I couldn't find any configs to set it up. The problem is if the model itself has input size w/h = 1280 x 1280 (instead of the default 640 x 640), the system tries to create a video stream of 1280 x 1280 (instead of 1280 x 720 in the default case), and constantly fails.

Model input size w/h: 1280 x 1280
choose_stream_resolution: VDO channel info contains no resolution info. Fallback to client-requested stream resolution.
choose_stream_resolution: We select stream w/h=1280 x 1280 based on VDO channel info.
Creating stream 1280 x 1280
Create stream failed: Invalid argument

[Cacsjep]

so you adjust the img provider to using a square 1280x1280 but got 1280x720 ? please clearify more =)

[hannaliavoshka]

Not exactly. We have two cases. In the first (successful) case, we use the Yolov5 tflite model, which has an input resolution 640x640. Loading it, as shown in the logs, the system creates a 1280x720 vdo stream. The model is loaded successfully. The logs are the following:
Model input size w/h: 640 x 640
choose_stream_resolution: We select stream w/h=1280 x 720 based on VDO channel info.
Creating VDO image provider and creating stream 1280 x 720
Model loaded successfully

The second and unsuccessful case is when I try to load Yolov5 tflite model with an input resolution 1280x1280. In this case, the system tries to create 1280x1280 vdo stream and fails with the error:
Model input size w/h: 1280 x 1280
choose_stream_resolution: VDO channel info contains no resolution info. Fallback to client-requested stream resolution.
choose_stream_resolution: We select stream w/h=1280 x 1280 based on VDO channel info.
Creating stream 1280 x 1280
Create stream failed: Invalid argument

[Cacsjep]

What camera model ? Did you have an camera with an vertical resolution higher as 1080 eg 1280 ?
I don't know your preprocessing but looks like you crop an higer stream to fit model input. Just for you to know you can request directly 640"640 eg square resolutions from VDO. In short I think your camera are not possible to create a 12801280 stream like may a full hd camera or may such high square reso not possible. I would directly create the stream with 12801280 and look for VDO errors

[hannaliavoshka]

You were right; the cause of the problem was that the camera's maximum video resolution is 1920x1080. Trying to run the model with an image size of 1280x1280 meant I was attempting to create a video stream of higher resolution than was possible. For instance, the model with a 1024x1024 image size is running successfully. Additionally, I was slightly confused by the error logs ('Invalid argument'), as the error message in this case should be 'Requested width or height larger than max resolution. Limit the requested resolution to max', which is what the imgprovider.c returns at first glance. Anyway, it is possible to run Yolov5 inference with an image size different from the default 640x640. For this purpose, a model should be exported in .tflite format with the specified resolution, which should comply with the camera resolution ability. Thanks for the help!

[Cacsjep]

@hannaliavoshka nice glad that it helps when you like we have a community discord when you want to join just ping me 😀.
Yeah a lot of us just used the examples to start and the img provider/auto resolution I don't like. Because when you need 640*640 just request that one and not checking for other and when this fails you have anyway a problem. I mean yolo would also work with lower resolutions but then you would need handling scaling correctly.