Extracting joint keypoint information to .json files (like seen generated in the DREAM project datasets)

[lakshmi-sathi]

Hi,

Thanks for such a wonderful tool offering to the research community. I wanted to extract the joint keypoint information of the Jaco-2 robot arm (3D and projected coordinates) into the .json files created during data capture.

I have imported the model of the Jaco2 arm (skeletal mesh) into the NDDS project in UE4 and have done some domain randomization. I have used NVCapturableActorTag with which I could extract the location and rotation of the arm model as a whole and also of the camera, into .json files with the images captured, however, I want to extract the keypoint information also for training keypoint detection.

I can see the joint keypoints getting represented with blue markers during the domain randomized data capture (below image).

but I would like to extract this keypoint information into the .json file as seen in the synthetic datasets in the DREAM project (below image)

What may be a way to proceed with this, is there any direct option in NDDS to extract this keypoint information to the .json file?

Thanks.

[wongsinglam]

Hi, I am quite interested in your keypoint in your first image. How do you mark your keypoint to the robot? And how do you configure your robot in the ue4 project, thanks

[hetolin]

@lakshmi-sathi hi, I am also quite interested in your keypoint in your first image.
How do you import the Jaco robot model? Do you import the urdf file or other types of files?
And after importing the model, how could you control the robot joints?