Comparison to YCB-Video Dataset

[lkaesberg]

Hi,
I am currently trying to create a python script which can evaluate the performance of the dope network with data from the YCB-Video dataset. But the rotation and translation vector differs quite a lot from what I get from the dataset. I think it is because dope calculates the translation and rotation of the cuboid and not of the base 3d model. Is there some easy way to change that or reverse the coordinates to fit to the base model?

In the picture, you can see that dope detects the object, but there is always an offset for the translation and rotation vector.
Kind regards
Lars

[TontonTremblay]

We changed the scale and the center of the YCB objects. I think this is what you are seeing. We saved the transforms we did. https://github.com/NVIDIA/Dataset_Utilities#nvdu_ycb check the transform we applied. I believe I also have them in some file. Let me find them for you.

[lkaesberg]

https://github.com/NVIDIA/Dataset_Utilities/blob/master/nvdu/config/object_settings/_ycb_original.json I think these should be the transformations I need right?

[TontonTremblay]

yes this will put the ycb object into the ycb modified. I just remember that Yu did change the reference frames in ycb videos. I have these transforms, going from ycb_dope to ycb_videos?

…

On Tue, Mar 22, 2022 at 8:30 AM Lars Kaesberg ***@***.***> wrote: https://github.com/NVIDIA/Dataset_Utilities/blob/master/nvdu/config/object_settings/_ycb_original.json I think these should be the transformations I need right? — Reply to this email directly, view it on GitHub <#230 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABK6JIE4SNXPRFNOXJEIG7LVBHRRJANCNFSM5QYLREXA> . You are receiving this because you commented.Message ID: ***@***.***>

[lkaesberg]

Yes YCB dope to YCB Video

[TontonTremblay]

I cannot find it, but I remember Yunzhi Lin computed them. I will share this thread with him if he still has them somewhere.

[TontonTremblay]

Transform_video2dope.csv

I think this is what you need.

[lkaesberg]

Okay thank you, but that is video to dope. Do I just have to multiply the dope quaternions with the inverse of the quaternion from the csv file, or how do I reverse that?

[lkaesberg]

I currently achieved a good rotation vector, but the object is flipped upside down. And I have to divide the translation vector by 10. I think the small offset is there because of the flipped rotation vector. If you have any idea what I am doing wrong please let me know

Code:

dope_tvec = np.add(self.base_translation, dope_tvec)
dope_tvec = dope_tvec / 10
dope_quat = quaternion_multiply(quaternion.inverse(self.base_rotation), dope_quat)
dope_rvec = Rotation.from_quat(dope_quat).as_rotvec()