Skip to content

Add body keyframe observation support in AMPLoader#67

Open
GiulioRomualdi wants to merge 5 commits into
mainfrom
keyframe
Open

Add body keyframe observation support in AMPLoader#67
GiulioRomualdi wants to merge 5 commits into
mainfrom
keyframe

Conversation

@GiulioRomualdi

@GiulioRomualdi GiulioRomualdi commented Jun 1, 2026

Copy link
Copy Markdown
Collaborator

Introduce support for body keyframe observations in AMPLoader, along with corresponding tests to ensure the functionality works as expected.

To use this feature, the user needs to specify the following dataset_cfg in the agent configuration:

dataset_cfg = {
    "amp_data_path": "/path/to/datasets",
    "datasets": {"walk_motion": 1.0},
    "slow_down_factor": 1,
    "amp_obs_components": ["joint_pos", "joint_vel", "base_lin_vel", "base_ang_vel", "body_pos_b", "body_ori_b"],
    "body_links_names": ["root_link", "left_knee", "right_knee", "left_foot", "right_foot"],
    "anchor_body_name": "root_link",
}

while, in the environment configuration:

@configclass
class AMPCfg(ObsGroup):
    joint_pos = ObsTerm(func=mdp.joint_pos)
    joint_vel = ObsTerm(func=mdp.joint_vel)
    base_lin_vel = ObsTerm(func=mdp.base_lin_vel)
    base_ang_vel = ObsTerm(func=mdp.base_ang_vel)
    body_pos_b = ObsTerm(func=gb_mdp.amp_body_pos_b, params={
        "anchor_body_name": "root_link",
        "body_names": ["root_link", "left_knee", "right_knee", "left_foot", "right_foot"],
    })
    body_ori_b = ObsTerm(func=gb_mdp.amp_body_ori_b, params={
        "anchor_body_name": "root_link",
        "body_names": ["root_link", "left_knee", "right_knee", "left_foot", "right_foot"],
    })

The user must ensure that amp_obs_components and the observation group produce the same observation vector. Moreover, the .npy prior should contain the required keyframe poses.

Notice that:

  1. If specified, the keyframes are used to compute the relative transformation between the root link and each keyframe.
  2. The orientation is provided to the discriminator network as a 6D vector, as suggested in On the Continuity of Rotation Representations in Neural Networks.

Important

The code is backward compatible: if amp_obs_components is not provided, the previous observation components will be used.

Tip

This feature can be used to enforce tracking of specific robot frames, such as end effectors.

@GiulioRomualdi GiulioRomualdi self-assigned this Jun 1, 2026
@GiulioRomualdi

Copy link
Copy Markdown
Collaborator Author

This can be interesting for @gbionics/team-hermes and @gbionics/team-dexter

@LoreMoretti LoreMoretti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Just a minor comment.

body_pos_b = rel_pos_b_flat
body_ori_b = rel_ori_b_flat
body_lin_vel_b = self._compute_raw_derivative(rel_pos_b_flat, simulation_dt)
body_ang_vel_b = self._compute_raw_derivative(rel_ori_b_flat, simulation_dt)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't _compute_ang_vel more coherent here? In case, we should provide the full rotation matrix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants