fmb
Stay organized with collections
Save and categorize content based on your preferences.
Our dataset consists of objects in diverse appearance and geometry. It requires
multi-stage and multi-modal fine motor skills to successfully assemble the pegs
onto a unfixed board in a randomized scene. We collected a total of 22,550
trajectories across two different tasks on a Franka Panda arm. We record the
trajectories from 2 global views and 2 wrist views. Each view contains both RGB
and depth map.
| Split |
Examples |
'train' |
1,804 |
FeaturesDict({
'episode_metadata': FeaturesDict({
'episode_language_embedding': Tensor(shape=(512,), dtype=float32),
'episode_language_instruction': string,
'episode_task': string,
'file_path': string,
}),
'steps': Dataset({
'action': Tensor(shape=(7,), dtype=float32),
'discount': Scalar(shape=(), dtype=float32),
'is_first': bool,
'is_last': bool,
'is_terminal': bool,
'language_embedding': Tensor(shape=(512,), dtype=float32),
'language_instruction': string,
'observation': FeaturesDict({
'color_id': Scalar(shape=(), dtype=uint8),
'eef_force': Tensor(shape=(3,), dtype=float32),
'eef_pose': Tensor(shape=(7,), dtype=float32),
'eef_torque': Tensor(shape=(3,), dtype=float32),
'eef_vel': Tensor(shape=(6,), dtype=float32),
'image_side_1': Image(shape=(256, 256, 3), dtype=uint8),
'image_side_1_depth': Tensor(shape=(256, 256), dtype=float32),
'image_side_2': Image(shape=(256, 256, 3), dtype=uint8),
'image_side_2_depth': Tensor(shape=(256, 256), dtype=float32),
'image_wrist_1': Image(shape=(256, 256, 3), dtype=uint8),
'image_wrist_1_depth': Tensor(shape=(256, 256), dtype=float32),
'image_wrist_2': Image(shape=(256, 256, 3), dtype=uint8),
'image_wrist_2_depth': Tensor(shape=(256, 256), dtype=float32),
'joint_pos': Tensor(shape=(7,), dtype=float32),
'joint_vel': Tensor(shape=(7,), dtype=float32),
'length': string,
'object_id': Scalar(shape=(), dtype=uint8),
'primitive': string,
'shape_id': Scalar(shape=(), dtype=uint8),
'size': string,
'state_gripper_pose': Scalar(shape=(), dtype=float32),
}),
'reward': Scalar(shape=(), dtype=float32),
}),
})
| Feature |
Class |
Shape |
Dtype |
Description |
|
FeaturesDict |
|
|
|
| episode_metadata |
FeaturesDict |
|
|
|
| episode_metadata/episode_language_embedding |
Tensor |
(512,) |
float32 |
|
| episode_metadata/episode_language_instruction |
Tensor |
|
string |
|
| episode_metadata/episode_task |
Tensor |
|
string |
|
| episode_metadata/file_path |
Tensor |
|
string |
|
| steps |
Dataset |
|
|
|
| steps/action |
Tensor |
(7,) |
float32 |
|
| steps/discount |
Scalar |
|
float32 |
|
| steps/is_first |
Tensor |
|
bool |
|
| steps/is_last |
Tensor |
|
bool |
|
| steps/is_terminal |
Tensor |
|
bool |
|
| steps/language_embedding |
Tensor |
(512,) |
float32 |
|
| steps/language_instruction |
Tensor |
|
string |
|
| steps/observation |
FeaturesDict |
|
|
|
| steps/observation/color_id |
Scalar |
|
uint8 |
|
| steps/observation/eef_force |
Tensor |
(3,) |
float32 |
|
| steps/observation/eef_pose |
Tensor |
(7,) |
float32 |
|
| steps/observation/eef_torque |
Tensor |
(3,) |
float32 |
|
| steps/observation/eef_vel |
Tensor |
(6,) |
float32 |
|
| steps/observation/image_side_1 |
Image |
(256, 256, 3) |
uint8 |
|
| steps/observation/image_side_1_depth |
Tensor |
(256, 256) |
float32 |
|
| steps/observation/image_side_2 |
Image |
(256, 256, 3) |
uint8 |
|
| steps/observation/image_side_2_depth |
Tensor |
(256, 256) |
float32 |
|
| steps/observation/image_wrist_1 |
Image |
(256, 256, 3) |
uint8 |
|
| steps/observation/image_wrist_1_depth |
Tensor |
(256, 256) |
float32 |
|
| steps/observation/image_wrist_2 |
Image |
(256, 256, 3) |
uint8 |
|
| steps/observation/image_wrist_2_depth |
Tensor |
(256, 256) |
float32 |
|
| steps/observation/joint_pos |
Tensor |
(7,) |
float32 |
|
| steps/observation/joint_vel |
Tensor |
(7,) |
float32 |
|
| steps/observation/length |
Tensor |
|
string |
|
| steps/observation/object_id |
Scalar |
|
uint8 |
|
| steps/observation/primitive |
Tensor |
|
string |
|
| steps/observation/shape_id |
Scalar |
|
uint8 |
|
| steps/observation/size |
Tensor |
|
string |
|
| steps/observation/state_gripper_pose |
Scalar |
|
float32 |
|
| steps/reward |
Scalar |
|
float32 |
|
https://doi.org/10.48550/arXiv.2401.08553
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-05-31 UTC.
[null,null,["Last updated 2024-05-31 UTC."],[],[],null,["# fmb\n\n\u003cbr /\u003e\n\n- **Description**:\n\nOur dataset consists of objects in diverse appearance and geometry. It requires\nmulti-stage and multi-modal fine motor skills to successfully assemble the pegs\nonto a unfixed board in a randomized scene. We collected a total of 22,550\ntrajectories across two different tasks on a Franka Panda arm. We record the\ntrajectories from 2 global views and 2 wrist views. Each view contains both RGB\nand depth map.\n\n- **Homepage** :\n \u003chttps://functional-manipulation-benchmark.github.io/\u003e\n\n- **Source code** :\n [`tfds.robotics.rtx.Fmb`](https://github.com/tensorflow/datasets/tree/master/tensorflow_datasets/robotics/rtx/rtx.py)\n\n- **Versions**:\n\n - **`0.1.0`** (default): Initial release.\n- **Download size** : `Unknown size`\n\n- **Dataset size** : `356.63 GiB`\n\n- **Auto-cached**\n ([documentation](https://www.tensorflow.org/datasets/performances#auto-caching)):\n No\n\n- **Splits**:\n\n| Split | Examples |\n|-----------|----------|\n| `'train'` | 1,804 |\n\n- **Feature structure**:\n\n FeaturesDict({\n 'episode_metadata': FeaturesDict({\n 'episode_language_embedding': Tensor(shape=(512,), dtype=float32),\n 'episode_language_instruction': string,\n 'episode_task': string,\n 'file_path': string,\n }),\n 'steps': Dataset({\n 'action': Tensor(shape=(7,), dtype=float32),\n 'discount': Scalar(shape=(), dtype=float32),\n 'is_first': bool,\n 'is_last': bool,\n 'is_terminal': bool,\n 'language_embedding': Tensor(shape=(512,), dtype=float32),\n 'language_instruction': string,\n 'observation': FeaturesDict({\n 'color_id': Scalar(shape=(), dtype=uint8),\n 'eef_force': Tensor(shape=(3,), dtype=float32),\n 'eef_pose': Tensor(shape=(7,), dtype=float32),\n 'eef_torque': Tensor(shape=(3,), dtype=float32),\n 'eef_vel': Tensor(shape=(6,), dtype=float32),\n 'image_side_1': Image(shape=(256, 256, 3), dtype=uint8),\n 'image_side_1_depth': Tensor(shape=(256, 256), dtype=float32),\n 'image_side_2': Image(shape=(256, 256, 3), dtype=uint8),\n 'image_side_2_depth': Tensor(shape=(256, 256), dtype=float32),\n 'image_wrist_1': Image(shape=(256, 256, 3), dtype=uint8),\n 'image_wrist_1_depth': Tensor(shape=(256, 256), dtype=float32),\n 'image_wrist_2': Image(shape=(256, 256, 3), dtype=uint8),\n 'image_wrist_2_depth': Tensor(shape=(256, 256), dtype=float32),\n 'joint_pos': Tensor(shape=(7,), dtype=float32),\n 'joint_vel': Tensor(shape=(7,), dtype=float32),\n 'length': string,\n 'object_id': Scalar(shape=(), dtype=uint8),\n 'primitive': string,\n 'shape_id': Scalar(shape=(), dtype=uint8),\n 'size': string,\n 'state_gripper_pose': Scalar(shape=(), dtype=float32),\n }),\n 'reward': Scalar(shape=(), dtype=float32),\n }),\n })\n\n- **Feature documentation**:\n\n| Feature | Class | Shape | Dtype | Description |\n|-----------------------------------------------|--------------|---------------|---------|-------------|\n| | FeaturesDict | | | |\n| episode_metadata | FeaturesDict | | | |\n| episode_metadata/episode_language_embedding | Tensor | (512,) | float32 | |\n| episode_metadata/episode_language_instruction | Tensor | | string | |\n| episode_metadata/episode_task | Tensor | | string | |\n| episode_metadata/file_path | Tensor | | string | |\n| steps | Dataset | | | |\n| steps/action | Tensor | (7,) | float32 | |\n| steps/discount | Scalar | | float32 | |\n| steps/is_first | Tensor | | bool | |\n| steps/is_last | Tensor | | bool | |\n| steps/is_terminal | Tensor | | bool | |\n| steps/language_embedding | Tensor | (512,) | float32 | |\n| steps/language_instruction | Tensor | | string | |\n| steps/observation | FeaturesDict | | | |\n| steps/observation/color_id | Scalar | | uint8 | |\n| steps/observation/eef_force | Tensor | (3,) | float32 | |\n| steps/observation/eef_pose | Tensor | (7,) | float32 | |\n| steps/observation/eef_torque | Tensor | (3,) | float32 | |\n| steps/observation/eef_vel | Tensor | (6,) | float32 | |\n| steps/observation/image_side_1 | Image | (256, 256, 3) | uint8 | |\n| steps/observation/image_side_1_depth | Tensor | (256, 256) | float32 | |\n| steps/observation/image_side_2 | Image | (256, 256, 3) | uint8 | |\n| steps/observation/image_side_2_depth | Tensor | (256, 256) | float32 | |\n| steps/observation/image_wrist_1 | Image | (256, 256, 3) | uint8 | |\n| steps/observation/image_wrist_1_depth | Tensor | (256, 256) | float32 | |\n| steps/observation/image_wrist_2 | Image | (256, 256, 3) | uint8 | |\n| steps/observation/image_wrist_2_depth | Tensor | (256, 256) | float32 | |\n| steps/observation/joint_pos | Tensor | (7,) | float32 | |\n| steps/observation/joint_vel | Tensor | (7,) | float32 | |\n| steps/observation/length | Tensor | | string | |\n| steps/observation/object_id | Scalar | | uint8 | |\n| steps/observation/primitive | Tensor | | string | |\n| steps/observation/shape_id | Scalar | | uint8 | |\n| steps/observation/size | Tensor | | string | |\n| steps/observation/state_gripper_pose | Scalar | | float32 | |\n| steps/reward | Scalar | | float32 | |\n\n- **Supervised keys** (See\n [`as_supervised` doc](https://www.tensorflow.org/datasets/api_docs/python/tfds/load#args)):\n `None`\n\n- **Figure**\n ([tfds.show_examples](https://www.tensorflow.org/datasets/api_docs/python/tfds/visualization/show_examples)):\n Not supported.\n\n- **Examples**\n ([tfds.as_dataframe](https://www.tensorflow.org/datasets/api_docs/python/tfds/as_dataframe)):\n\nDisplay examples... \n\n- **Citation**:\n\n https://doi.org/10.48550/arXiv.2401.08553"]]
