arc

  • Description:

ARC can be seen as a general artificial intelligence benchmark, as a program synthesis benchmark, or as a psychometric intelligence test. It is targeted at both humans and artificially intelligent systems that aim at emulating a human-like form of general fluid intelligence.

Split Examples
'test' 400
'train' 400
  • Feature structure:
FeaturesDict({
    'task_id': Text(shape=(), dtype=string),
    'test': Sequence({
        'input': Sequence(Sequence(int32)),
        'output': Sequence(Sequence(int32)),
    }),
    'train': Sequence({
        'input': Sequence(Sequence(int32)),
        'output': Sequence(Sequence(int32)),
    }),
})
  • Feature documentation:
Feature Class Shape Dtype Description
FeaturesDict
task_id Text string
test Sequence
test/input Sequence(Sequence(Tensor)) (None, None) int32
test/output Sequence(Sequence(Tensor)) (None, None) int32
train Sequence
train/input Sequence(Sequence(Tensor)) (None, None) int32
train/output Sequence(Sequence(Tensor)) (None, None) int32
  • Citation:
@misc{chollet_francois_2019,
  title     = {The Measure of Intelligence},
  url       = {https://arxiv.org/abs/1911.01547},
  journal   = {arXiv.org},
  author    = {Francois Chollet},
  year      = {2019},
  month     = {Nov}
}

arc/2019-12-06 (default config)