Data Workflow#

This page describes FGManip's unified data workflow for fine-grained manipulation: Record -> Merge -> Replay -> Convert.

The same trajectory assets can be reused across algorithms and adapted to external toolchains such as LeRobot and StarVLA.

Data Recording#

Collect trajectories with motion planning and FGManip callable skills. For fast collection, you can record with obs_mode=none and add observations later in replay.

python record.py --env-id grasp_part --skill-type grasp_part \
  --object-name bottle --part-name cap \
  -n 10 --record-dir demos --save-video

Trajectory Replay#

Replay trajectories with new observation/control settings and optionally save new trajectories/videos. Use --use-first-env-state for aligned initial states.

python utils/replay_trajectory.py \
  --traj-path demos/trajectory.h5 \
  --use-first-env-state -c pd_joint_delta_pos -o rgb \
  --save-traj --num-envs 10 -b physx_cpu

Rendering quality options: --shader rt (high quality) or --shader rt-fast (faster).

Dataset Merge#

Merge multiple trial outputs into one HDF5 trajectory file. By default, only successful trajectories are merged.

python utils/merge_trajectory.py -i /nat/demos/grasp_part/grasp_part_1 \
  -o /nat/demos/grasp_part/grasp_part_1/trajectory.h5 \
  -p trajectory.h5

Format Conversion#

Convert merged or replayed trajectories to downstream training formats. The example below converts to LeRobot format.

python utils/convert_to_lerobot.py \
  --traj-path demos/trajectory.h5 \
  --output-dir demos/lerobot_grasp_part \
  --task-name "Grasp Part" \
  --fps 30 \
  --robot-type panda

Validation example:

lerobot-dataset-viz --repo-id grasp_part --root demos/lerobot_grasp_part --mode local --episode-index 0

Workflow Notes#

  • Recommended order: Record -> Merge -> Replay -> Convert.

  • Replay is the key stage for changing observation/control modes without recollecting raw trajectories.

  • This workflow is designed for reproducible benchmark data generation across policies and codebases.