Thanks for tuning in to Google I/O. View all sessions on demandWatch on demand


SI-Score (Synthetic Interventions on Scenes for Robustness Evaluation) is a dataset to evaluate robustness of image classification models to changes in object size, location and rotation angle.

In SI-SCORE, we take objects and backgrounds and systematically vary object size, location and rotation angle so we can study the effect of changing these factors on model performance. The image label space is the ImageNet label space (1k classes) for easy evaluation of models.

More information about the dataset can be found at

    'dataset_label': ClassLabel(shape=(), dtype=int64, num_classes=1000),
    'image': Image(shape=(None, None, 3), dtype=uint8),
    'image_id': int64,
    'label': ClassLabel(shape=(), dtype=int64, num_classes=1000),
  • Feature documentation:
Feature Class Shape Dtype Description
dataset_label ClassLabel int64
image Image (None, None, 3) uint8
image_id Tensor int64
label ClassLabel int64
      title={On Robustness and Transferability of Convolutional Neural Networks},
      author={Josip Djolonga and Jessica Yung and Michael Tschannen and Rob Romijnders and Lucas Beyer and Alexander Kolesnikov and Joan Puigcerver and Matthias Minderer and Alexander D'Amour and Dan Moldovan and Sylvain Gelly and Neil Houlsby and Xiaohua Zhai and Mario Lucic},

siscore/rotation (default config)

  • Config description: factor of variation: rotation

  • Download size: 1.40 GiB

  • Dataset size: 1.40 GiB

  • Splits:

Split Examples
'test' 39,540



  • Config description: factor of variation: size

  • Download size: 3.25 GiB

  • Dataset size: 3.27 GiB

  • Splits:

Split Examples
'test' 92,884



  • Config description: factor of variation: location

  • Download size: 18.21 GiB

  • Dataset size: 18.31 GiB

  • Splits:

Split Examples
'test' 541,548