Recording demonstrations
record.py is the single entrypoint for generating expert demonstrations. It runs an MP-based solver (motion planning + scripted skill controllers) inside the SAPIEN simulator, then writes successful episodes to an HDF5 trajectory paired with video. Two modes: single-skill (an atomic-skill env) and --task-graph (a multi-stage / multi-task env).
Single-skill mode
Collect demos for one atomic-skill env against one asset/part — typical for skill-level data or ablations. Supply -e/--env-id (a registered env id), --object-name (the asset id) and --part-name (the target part). --only-count-success keeps generating until -n successful trajectories are written.
# Record 5 successful grasps on asset 100221's "cap" part $ python record.py -e grasp_part --object-name 100221 --part-name cap -n 5 --only-count-success # Open the SAPIEN viewer + ray-traced shader for inspection $ python record.py -e toggle_switch --object-name 100920 --part-name switch \ -n 5 --vis --shader rt-fast
--task-graph mode
For compositional tasks, point --task-graph at a YAML chain (this drives a multi-stage / multi-task env). Each stage is a skill call against a named asset/part; per-stage predicates decide success. Only successful trajectories are kept when --only-count-success is set.
# Record 5 successful "grasp + lift cap" composite demos $ python record.py --task-graph configs/example_grasp_cap.yaml -n 5 --only-count-success # Custom output dir; CPU multiprocessing for faster collection $ python record.py --task-graph configs/lid_opening_100221.yaml \ -n 50 --record-dir demos/lid_v2 --num-procs 4 -b physx_cpu
How the expert solver works
Each atomic skill is paired with a scripted controller that uses motion planning for free-space motion and pre-programmed contact-phase trajectories for the engagement step. Failure cases (unreachable target, IK fail, predicate not met) are retried; with --only-count-success the recorder keeps going until -n successes land on disk.
Output layout
Recording writes one sub-directory per trial under --record-dir (default demos/):
demos/
<env>/
trial_0001/
trajectory.h5 # ManiSkill RecordEpisode HDF5 (actions + env states)
trajectory.json # episode metadata (env id, asset, control_mode, success)
trial_0002/
trajectory.h5
trajectory.json
0.mp4 # only present on successful trials
With --only-count-success the recorder retries until -n trials succeed; failed attempts leave a trial_NNNN/ dir with an empty trajectory.h5. Recording stores actions + env states only — observations are rendered later by replay. Run merge then replay (point merge -i at the <env>/ dir; it recurses into the trial_* dirs) before converting.
Common flags
| Flag | Default | Notes |
|---|---|---|
-e, --env-id | peg_in_hole | Registered env id (single-skill mode) |
--object-name | 100937 | Asset id |
--part-name | button | Target part on the asset |
--task-graph | — | Path to a task-graph YAML (composite / multi-task mode) |
-n, --num-traj | 5 | Number of trajectories to generate |
--only-count-success | off | Keep generating until -n successes; only save successes |
-b, --sim-backend | auto | auto / physx_cpu / gpu |
--render-mode | rgb_array | rgb_array or sensors (affects saved video) |
--vis | off | Open the SAPIEN GUI to watch live |
--shader | default | default (fast) · rt / rt-fast (ray-traced) |
--record-dir | demos | Output root |
--traj-name | trajectory | Base name of the .h5 |
--num-procs | 1 | CPU multiprocessing (CPU backend only) |
Canonical source: python record.py --help.