Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LEAP hand. #91

Merged
merged 6 commits into from
Sep 6, 2024
Merged

Conversation

kevinzakka
Copy link
Collaborator

@kevinzakka kevinzakka commented Aug 28, 2024

Fixes #84.

leap_hand

@kevinzakka kevinzakka requested a review from yuvaltassa August 28, 2024 01:14
@kevinzakka kevinzakka mentioned this pull request Aug 28, 2024
@kevinzakka
Copy link
Collaborator Author

cc @Lucky7Chess

@Lucky7Chess
Copy link

Lucky7Chess commented Aug 28, 2024

Yes, the right hand looks good to me @kevinzakka I just loaded it. Maybe we should also add the left hand also from https://leaphand.com/static/cad/urdf/leap_left.zip ?

I'll add a link to your file on menagerie to the LEAP Hand website.

@kevinzakka
Copy link
Collaborator Author

Awesome, thanks so much for fast review @Lucky7Chess! Agree having the left hand would be good. Is it as simple as reflecting the meshes about the z axis? I'm trying to see if we can write a script to generate the left from right to minimize bugs.

@kevinzakka
Copy link
Collaborator Author

I noticed the following differences between the left and right models:

  • The left hand references a thumb_left_temp_base whereas this does not exist in the right hand.
  • No pip_4 in the left hand.
  • Different ordering of joint names in the thumb, e.g. right order is [12, 13, 14, 15] whereas left order is [13, 12, 14, 15].
  • It would seem that the meshes don't have the same origin? See the difference when the two hands are placed the world origin (pic below).

Just want to make sure that these are intentional and wanted to walk through them with you @Lucky7Chess.

left right

@Lucky7Chess
Copy link

Sorry for some reason Github didn't notify me of your message.

Yes I wish there was an easy way to mirror but there is not it seems. But I think the model isn't an exact mirror in this case.

None of these three things you mention are intentional actually. The first two is just some CAD software naming I didn't change. And the last is just how my CAD software exports and I never fixed it to a set position. Its just a byproduct of the software I am using to get the URDF. Its a messy mix of Inventor/Onshape.

I guess I don't use the left hand much so I haven't noticed the differences. Its probably better to fix these bugs in the left on Menagerie since that'll be what people use on MuJoCo.

@kevinzakka
Copy link
Collaborator Author

kevinzakka commented Aug 28, 2024

@Lucky7Chess What does thumb_left_temp_base correspond to? I tried replacing it with pip and adjusting the transform but struggling to match the right hand.

Was able to figure this out!

@kevinzakka
Copy link
Collaborator Author

@Lucky7Chess One last thing: I want to add an attachment site to each hand so that users can use it to attach the hand to an arm. Do you have a pos/quat that I can use for this for each hand, potentially from CAD? I couldn't find anything from parsing the original URDF.

@Lucky7Chess
Copy link

Hmm I actually never thought about adding that info to the URDF/MJCF but that's a really good idea. I can't figure out how to get the coordinates from the URDF in the format you may want but I can get it in my CAD. The center of the mounting holes is where the center of the arm end effector mounting point would contact the hand. As in the photo below that is 37.5mm from the top of the top plate and 59mm from the flat side of the top plate.

image

The top plate is the same for the right hand (the plate you are looking at on the image attached) and the left hand except it is mirrored. So for the left hand it is 37.5 mm from the top and 59mm from the right side which is still the flat side of the top palm.

If this doesn't make sense I can also try to figure this out in URDF somehow? I am unfamiliar with the easiest way to do that. Thank you for your help, you are an expert in this! :)

@Lucky7Chess
Copy link

It also looks like I didn't put the mounting holes into the URDF, not sure why. The hand typically mounts to UR5/Xarm/Franka that's why its mounted to the top of the hand to avoid the arm colliding with the table due to the kinematic structure of these arms being designed for grippers that are also top down.

@kevinzakka
Copy link
Collaborator Author

kevinzakka commented Aug 28, 2024

@Lucky7Chess Maybe we can have the mounting plate as part of the URDF? If it's arm specific then maybe not. But I need the coordinates of the center in the palm frame if possible otherwise I can visually guesstimate the location!

@Lucky7Chess
Copy link

Lucky7Chess commented Aug 28, 2024

The plate is arm specific depending on the hole pattern of the arm...so the plate you have on the urdf is just a generic one without any holes. I guess now that I think of it the mounting position could also change slightly depending on the arm you have.

You could leave the mounting point off I guess or just pick a center location on the top of the palm, either way is fine. The point that I sent you is roughly aligned with the middle finger and slightly less than halfway down from the start of the middle finger/top from that image viewpoint.

@kevinzakka
Copy link
Collaborator Author

I think I’ll go with the latter option!

The only thing is that the estimates for left and right are different because the palm meshes seem to have a different coordinate frame. I want to figure out how to make them symmetric 🥲

@Lucky7Chess
Copy link

At one point I was trying to make them symmetric also but then I gave up I remember. The CAD -> URDF exporters I have all suck to be honest and there was no easy way for me to do this.

@Lucky7Chess
Copy link

I think its okay because as the end user its not really noticable how the meshes are setup.

@kevinzakka
Copy link
Collaborator Author

Thanks @Lucky7Chess, decided not to add an attachment site for now. We can add it in a future PR once we also validate with actual hardware!

@kevinzakka
Copy link
Collaborator Author

@yuvaltassa Ready for import now!

@Lucky7Chess
Copy link

Thank you so much @kevinzakka !!!

@kevinzakka
Copy link
Collaborator Author

I didn't do much, thank you for your awesome open-source contributions :)

@copybara-service copybara-service bot merged commit adbdab0 into google-deepmind:main Sep 6, 2024
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

LEAP HAND SUPPORT
3 participants