Part-Annotated Object Library

Part-level annotations—labeled parts and grasp pose candidates—form the foundation of FGManip's fine-grained manipulation. The library is extensible: you can add new objects and parts using the annotation tool, and the resulting grasp data feeds directly into the Skill Library for manipulation policies. This page describes the current scale of the library and how to use the annotation tool.

URDF structure for part-annotated objects.

Library Scale

Metric	Count
Annotated assets	431
Annotated parts	1,078
Grasp pose candidates	4,312

Annotation Tool

Average annotation time per grasp pose candidate: ~20 seconds.

The figure below shows the annotation tool interface:

Annotation tool interface for part and grasp pose labeling.

Run the tool to enter the annotation interface. The toolbar provides the following operations:

Toolbar Operations

save

Save the current annotations.

create

Create a new grasp pose. A grasp pose is created and can be moved in the viewport to place it, then adjusted to refine the grasp orientation.

edit

Edit an existing grasp pose. Use keyboard controls to move the selected pose.

remove

Remove the specified point or grasp pose.

list

List grasp points under each part.

resize

Adjust the URDF scale of the object.

After annotation, the tool generates a JSON file in the following format:

{
    "object_name": "bottle",
    "scale": 0.2,
    "grasp_parts": {
        "cap": [
            {
                "id": 0,
                "matrix": [[-0.996, 0, 0.087, 0.031], [0, 1, 0, 0], [-0.087, 0, -0.996, 0.843], [0, 0, 0, 1]],
                "base": "base",
                "description": ""
            },
            {
                "id": 1,
                "matrix": [[-0.416, 0.893, 0.174, 0.019], [0.906, 0.423, 0, 0.05], [-0.073, 0.157, -0.985, 0.906], [0, 0, 0, 1]],
                "base": "base",
                "description": ""
            }
        ],
        "body": [
            {
                "id": 0,
                "matrix": [[-0.087, 0, 0.996, -0.037], [0, 1, 0, 0], [-0.996, 0, -0.087, 0.154], [0, 0, 0, 1]],
                "base": "base",
                "description": ""
            },
            {
                "id": 1,
                "matrix": [[-0.087, 0.996, 0, 0.007], [0, 0, 1, -0.05], [0.996, 0.087, 0, 0.248], [0, 0, 0, 1]],
                "base": "base",
                "description": ""
            },
            {
                "id": 2,
                "matrix": [[-0.301, 0.951, -0.072, -0.109], [0.316, 0.171, 0.933, -0.093], [0.900, 0.258, -0.352, 0.278], [0, 0, 0, 1]],
                "base": "base",
                "description": ""
            }
        ]
    }
}