Skip to content

so101-grasping.md

Latest commit

 

History

History
50 lines (37 loc) · 2.15 KB

so101-grasping.md

File metadata and controls

50 lines (37 loc) · 2.15 KB
Raw

VLA: SO-101 grasping

Give a SO-101 arm the instruction “Pick up the cube and place it in the bowl.” The π0.5 policy predicts actions from front and wrist camera images and joint state. EmbodiRun collects the observations, requests an action chunk, validates it, and executes it through the robot adapter.

Your browser does not support the video tag.

Three SO-101 recordings of the cube-to-bowl task, shown side by side.

Observation to action

  1. Capture front and wrist images together with the six-dimensional joint and gripper state.
  2. Send the observation and instruction to the π0.5 service.
  3. Map the returned action chunk to the robot, check limits, and play 50 steps at 20 Hz.
  4. Collect the next observation and repeat.

The Recipe uses one arm and an EmbodiInfer HTTP service. The model checkpoint must match the robot's camera roles, feature names, and action units.

Run the Recipe

Follow the SO-101 grasping Recipe to configure the robot, cameras, calibration, checkpoint, and SSH hosts.

Preview the shipped configuration from the repository root:

uv sync --frozen
bash examples/run.sh examples/engine_comparison/embodiinfer-http.yaml validate
bash examples/run.sh examples/engine_comparison/embodiinfer-http.yaml plan

These commands check the configuration and display the launch plan locally. The Recipe provides the remaining setup, startup, execution, and shutdown commands. Read Safety before enabling hardware motion.

Extend the setup