Join TensorFlow at Google I/O, May 11-12 Register now

rlu_dmlab_seekavoid_arena01

  • Description:

RL Unplugged is suite of benchmarks for offline reinforcement learning. The RL Unplugged is designed around the following considerations: to facilitate ease of use, we provide the datasets with a unified API which makes it easy for the practitioner to work with all data in the suite once a general pipeline has been established.

The datasets follow the RLDS format to represent steps and episodes.

DeepMind Lab dataset has several levels from the challenging, partially observable Deepmind Lab suite. DeepMind Lab dataset is collected by training distributed R2D2 by Kapturowski et al., 2018 agents from scratch on individual tasks. We recorded the experience across all actors during entire training runs a few times for every task. The details of the dataset generation process is described in Gulcehre et al., 2021.

We release datasets for five different DeepMind Lab levels: seekavoid_arena_01, explore_rewards_few, explore_rewards_many, rooms_watermaze, rooms_select_nonmatching_object. We also release the snapshot datasets for seekavoid_arena_01 level that we generated the datasets from a trained R2D2 snapshot with different levels of epsilons for the epsilon-greedy algorithm when evaluating the agent in the environment.

DeepMind Lab dataset is fairly large-scale. We recommend you to try it if you are interested in large-scale offline RL models with memory.

FeaturesDict({
    'episode_id': tf.int64,
    'episode_return': tf.float32,
    'steps': Dataset({
        'action': tf.int64,
        'discount': tf.float32,
        'is_first': tf.bool,
        'is_last': tf.bool,
        'is_terminal': tf.bool,
        'observation': FeaturesDict({
            'last_action': tf.int64,
            'last_reward': tf.float32,
            'pixels': Image(shape=(72, 96, 3), dtype=tf.uint8),
        }),
        'reward': tf.float32,
    }),
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
episode_id Tensor tf.int64
episode_return Tensor tf.float32
steps Dataset
steps/action Tensor tf.int64
steps/discount Tensor tf.float32
steps/is_first Tensor tf.bool
steps/is_last Tensor tf.bool
steps/is_terminal Tensor tf.bool
steps/observation FeaturesDict
steps/observation/last_action Tensor tf.int64
steps/observation/last_reward Tensor tf.float32
steps/observation/pixels Image (72, 96, 3) tf.uint8
steps/reward Tensor tf.float32
@article{gulcehre2021rbve,
    title={Regularized Behavior Value Estimation},
    author={ {\c{C} }aglar G{\"{u} }l{\c{c} }ehre and
               Sergio G{\'{o} }mez Colmenarejo and
               Ziyu Wang and
               Jakub Sygnowski and
               Thomas Paine and
               Konrad Zolna and
               Yutian Chen and
               Matthew W. Hoffman and
               Razvan Pascanu and
               Nando de Freitas},
    year={2021},
    journal   = {CoRR},
    url       = {https://arxiv.org/abs/2103.09575},
    eprint={2103.09575},
    archivePrefix={arXiv},
}

rlu_dmlab_seekavoid_arena01/training_0 (default config)

  • Dataset size: 381.13 GiB

  • Splits:

Split Examples
'train' 134,707

rlu_dmlab_seekavoid_arena01/training_1

  • Dataset size: 360.77 GiB

  • Splits:

Split Examples
'train' 128,472

rlu_dmlab_seekavoid_arena01/training_2

  • Dataset size: 378.58 GiB

  • Splits:

Split Examples
'train' 133,545

rlu_dmlab_seekavoid_arena01/snapshot_0_eps_0.0

  • Dataset size: 94.83 GiB

  • Splits:

Split Examples
'train' 33,340

rlu_dmlab_seekavoid_arena01/snapshot_1_eps_0.0

  • Dataset size: 94.68 GiB

  • Splits:

Split Examples
'train' 33,340

rlu_dmlab_seekavoid_arena01/snapshot_0_eps_0.01

  • Dataset size: 94.77 GiB

  • Splits:

Split Examples
'train' 33,340

rlu_dmlab_seekavoid_arena01/snapshot_1_eps_0.01

  • Dataset size: 94.65 GiB

  • Splits:

Split Examples
'train' 33,340

rlu_dmlab_seekavoid_arena01/snapshot_0_eps_0.25

  • Dataset size: 93.96 GiB

  • Splits:

Split Examples
'train' 33,340

rlu_dmlab_seekavoid_arena01/snapshot_1_eps_0.25

  • Dataset size: 93.93 GiB

  • Splits:

Split Examples
'train' 33,340