Add keyboard shortcuts, body selection, and drag perturbation#21
Add keyboard shortcuts, body selection, and drag perturbation#21kevinzakka wants to merge 6 commits into
Conversation
Drive a mujoco.MjvPerturb with mjv_applyPerturbForce / mjv_applyPerturbPose so the spring force, critical damping (via mj_objectVelocity), and pose math match MuJoCo's simulate and studio. Input is web-native via viser's absolute drag target: cmd/ctrl+left-drag pulls the grab point toward the cursor (off-center grabs rotate through the moment arm) with a connector line, cmd/ctrl+right-drag rotates toward a screen-built orientation shown by a ghost box, and while paused both move free-joint and mocap bodies kinematically with mj_forward so contacts and forces refresh live. Also: number keys toggle geom groups and shift+number toggles site groups, camera tracking defaults off, and untracking shifts the camera so the view no longer teleports.
kevinzakka
force-pushed
the
kz/interactive-perturbation
branch
from
a59ff07 to
d73aed2
Compare
…keys Remove the handler's unused MjData reference (it always operates on the data passed in per call, and the stored copy was the scene's, not the viewer's stepped data). Factor the duplicated drag-state reset out of clear() and _end_drag(), and stop computing the grab-point selpos on every running-mode apply() since only the paused branch uses it. In the scene, initialize the camera dropdown attribute in __init__ instead of guarding with hasattr, and drop the redundant server argument from register_keybindings (the scene already holds it). Remove the shift+digit site-group hotkeys: viser matches hotkeys on event.key, where shift+1 is "!" not "1", so they never fired. The geom-group digit keys stay; sites remain toggleable via the Groups GUI checkboxes. The now-unused site-group checkbox dict goes with them. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
I am very much looking forward to this. Are there plans to integrate an xy plane drag that allows the robot to be repositioned without changing its height or rotation? Other simulators features this and I think it is also missing from the official mujoco viewer. |
|
@Flova it actually does exist in the native viewer but it's modifier-gated. The blocker is on the viser side, not MuJoCo: viser only projects the cursor onto a camera-aligned plane and freezes the drag modifier at drag-start, so we can't reproduce the "Shift switches to the world-horizontal plane mid-drag" behavior. We'd need viser to expose the drag-plane normal (or allow live modifiers during a drag). cc @brentyi |
|
Live modifiers: doable! |
Maybe I am missing something, but I tried this in the native viewer and while the force vector stays in the plane, it only applies a sidewards force, which is not that great if you e.g. want to reposition a humanoid in a (maybe also stopped) simulator without messing with its stability. Webots for example supports both force/torque based interaction and "absolute" changing of the environment relative to the ground plane. It would be really great to have a feature like this for higher level robot behavior tests with mujoco. |
3 participants
mjviser_drag.mp4
Adds MuJoCo-style interactive perturbation.
cmd/ctrl + left-drag springs the grabbed point toward the cursor, and grabbing off-center also rotates the body through the moment arm. cmd/ctrl + right-drag rotates the body toward the drag, with a wireframe ghost box showing the target orientation. While paused, both gestures pose free-joint and mocap bodies kinematically, with contacts and forces refreshed live.
The dynamics are MuJoCo's own: it drives an mjvPerturb through mjv_applyPerturbForce and mjv_applyPerturbPose, so force, damping, and poses match simulate. Only the input is web-native, since viser gives an absolute world-space drag target instead of relative mouse motion. On macOS use cmd, since ctrl + left-drag opens the browser context menu.
Also adds keyboard shortcuts (space, N, backspace; C/F/J/I/V/M/U/T visualization flags; digits for geom groups, shift+digit for site groups) and a camera dropdown with tracking off by default. Untracking no longer teleports the view. Bumps idna and urllib3 to clear the open Dependabot alerts.
Tested: force, torque, and pose math; the drag and rotate state machine; the frame offset; and rebuild safety.