asimov_multimodal_manual_val

• Description:

(Image, context, instruction) triplets manually taken and written by humans while ensuring that the instruction desirability can only be determined by looking at the image (validation set)

• Homepage: https://asimov-benchmark.github.io/

• Source code: tfds.robotics.asimov.AsimovMultimodalManualVal

• Versions:

  • 0.1.0 (default): Initial release.

• Download size: Unknown size

• Dataset size: 240.71 MiB

• Auto-cached (documentation): Only when shuffle_files=False (val)

• Splits:

Split	Examples
'val'	59

• Feature structure:

FeaturesDict({
    'image': Image(shape=(None, None, 3), dtype=uint8),
    'instructions': Sequence({
        'context': Text(shape=(), dtype=string),
        'instruction': Text(shape=(), dtype=string),
        'prompt_with_constitution': Text(shape=(), dtype=string),
        'prompt_with_constitution_chain_of_thought': Text(shape=(), dtype=string),
        'prompt_with_constitution_chain_of_thought_antijailbreak': Text(shape=(), dtype=string),
        'prompt_with_constitution_chain_of_thought_antijailbreak_adversary': Text(shape=(), dtype=string),
        'prompt_with_constitution_chain_of_thought_antijailbreak_adversary_parts': Sequence(Text(shape=(), dtype=string)),
        'prompt_with_constitution_chain_of_thought_antijailbreak_parts': Sequence(Text(shape=(), dtype=string)),
        'prompt_with_constitution_chain_of_thought_parts': Sequence(Text(shape=(), dtype=string)),
        'prompt_with_constitution_parts': Sequence(Text(shape=(), dtype=string)),
        'prompt_without_constitution': Text(shape=(), dtype=string),
        'prompt_without_constitution_parts': Sequence(Text(shape=(), dtype=string)),
        'undesirable_groundtruth_answer': bool,
    }),
    'original_image': Image(shape=(None, None, 3), dtype=uint8),
})

• Feature documentation:

Feature	Class	Shape	Dtype	Description
	FeaturesDict
image	Image	(None, None, 3)	uint8
instructions	Sequence
instructions/context	Text		string
instructions/instruction	Text		string
instructions/prompt_with_constitution	Text		string
instructions/prompt_with_constitution_chain_of_thought	Text		string
instructions/prompt_with_constitution_chain_of_thought_antijailbreak	Text		string
instructions/prompt_with_constitution_chain_of_thought_antijailbreak_adversary	Text		string
instructions/prompt_with_constitution_chain_of_thought_antijailbreak_adversary_parts	Sequence(Text)	(None,)	string
instructions/prompt_with_constitution_chain_of_thought_antijailbreak_parts	Sequence(Text)	(None,)	string
instructions/prompt_with_constitution_chain_of_thought_parts	Sequence(Text)	(None,)	string
instructions/prompt_with_constitution_parts	Sequence(Text)	(None,)	string
instructions/prompt_without_constitution	Text		string
instructions/prompt_without_constitution_parts	Sequence(Text)	(None,)	string
instructions/undesirable_groundtruth_answer	Tensor		bool
original_image	Image	(None, None, 3)	uint8

• Supervised keys (See as_supervised doc): None

• Figure (tfds.show_examples): Not supported.

• Examples (tfds.as_dataframe):

• Citation:

@article{sermanet2025asimov,
  author    = {Pierre Sermanet and Anirudha Majumdar and Alex Irpan and Dmitry Kalashnikov and Vikas Sindhwani},
  title     = {Generating Robot Constitutions & Benchmarks for Semantic Safety},
  journal   = {arXiv preprint arXiv:2503.08663},
  url       = {https://arxiv.org/abs/2503.08663},
  year      = {2025},
}