OpenVLA-OFT

This page describes how to evaluate OpenVLA-OFT policies in FGManip/ManiSkill environments.

Integration status: OpenVLA-OFT currently uses its own setup and evaluation scripts. Runtime conventions are aligned with the rest of this guide where possible, but this backend is not yet fully unified with all policy runners.

Quick Start

1. Setup Environment
Follow the setup section below.

2. Run Evaluation

bash core/policies/openvla-oft/eval.sh

Videos are saved to core/policies/openvla-oft/eval_videos/.

Usage Details

Main script: core/policies/openvla-oft/evaluate.py.

python core/policies/openvla-oft/evaluate.py \
  --pretrained_checkpoint <checkpoint_path_or_hub_id> \
  --task_name <task_name> \
  --num_episodes <N> \
  --save_video

Common arguments

• --task_name: task ID (see list below).

• --pretrained_checkpoint: local checkpoint path or Hugging Face model ID.

• --num_episodes: number of evaluation episodes.

• --save_video: save rollout videos under eval_videos/.

• --max_steps: max steps per episode (default: 200).

Supported Tasks

Task Name	Description
plug_charger	Pick up charger and plug into receptacle.
peg_in_hole	Insert peg into box with hole.
stand_up	Make object stand up.
toggle_switch	Toggle the switch.
toggle_switch_table	Toggle tabletop switch variant.
lid_opening	Open the bottle lid.
stack_pyramid	Stack into pyramid shape.
draw_triangle	Draw a triangle trajectory.
slide_along	Slide object along target path.
align_to_part	Align tool/object to a target part.

For full and up-to-date task mapping, check evaluate.py.

Setup Instructions

# directly overwrite the FGManip environment.
conda activate maniskill

# Install PyTorch (choose command for your machine from pytorch.org)
pip3 install torch torchvision torchaudio

# Clone openvla-oft repo and install dependencies
git clone https://github.com/moojink/openvla-oft.git
cd openvla-oft
pip install -e .

# Install Flash Attention 2 for training
pip install packaging ninja
ninja --version; echo $?   # should return 0
pip install "flash-attn==2.5.5" --no-build-isolation

If flash-attn install fails, try: pip cache remove flash_attn then reinstall.

Maintenance Checklist

• Keep task name mapping in this page synchronized with evaluate.py.

• Document any checkpoint format changes (local path vs hub ID).

• Keep video output path and argument defaults synchronized with eval.sh.