Image Rectification

[Nielsencu]

I have succesfully trained a model which can detect the orientation of the 3D model quite well.
https://user-images.githubusercontent.com/50238554/151923763-57abd341-871e-4665-aebb-0521ba86a6ba.mp4

However, the 3D centroid position in meters relative to the camera which is output by the network is still off, even though visually the 3D bounding box looks quite accurate. The issue might be with the deprojection by the PnP.

I am running the inference on intel realsense camera with fetches the camera intrinsics using the realsense API, and realsense documentation mentions that the camera by default should be well-calibrated with 0 distortion coeficients. The 3D bounding box of the model should also be correct because I got them from a 3D mesh which is hand-scanned. So the parameters passed to the PnP algorithm should be correct.

One possible explanation is #203 where it is suggested that image rectification should be done before feeding them to DOPE. Can you kindly provide a more detailed explanation on how should this image rectification be performed?

[mintar]

After you launch this file:

roslaunch realsense2_camera rs_rgbd.launch

... there is this topic:

/camera/color/image_rect_color

This is the rectified input topic which should be used as input to DOPE.