cerita_cloze

  • Deskripsi :

Story Cloze Test adalah kerangka penalaran akal sehat baru untuk mengevaluasi pemahaman cerita, pembuatan cerita, dan pembelajaran naskah. Tes ini membutuhkan sistem untuk memilih akhir cerita empat kalimat yang tepat.

  • Dokumentasi Tambahan : Jelajahi di Makalah Dengan Kode

  • Deskripsi konfigurasi : tahun 2018

  • Beranda : https://www.cs.rochester.edu/nlp/rocstories/

  • Kode sumber : tfds.datasets.story_cloze.Builder

  • Versi :

    • 1.0.0 (default): Rilis awal.

  • Ukuran unduhan : Unknown size

  • Instruksi pengunduhan manual : Kumpulan data ini mengharuskan Anda mengunduh data sumber secara manual ke download_config.manual_dir (default ke ~/tensorflow_datasets/downloads/manual/ ):
    Kunjungi https://www.cs.rochester.edu/nlp/rocstories/ dan isi formulir google untuk mendapatkan dataset. Anda akan menerima email dengan tautan untuk mengunduh kumpulan data. Untuk data 2016, file validasi dan pengujian perlu diganti namanya menjadi cloze_test val _spring2016.csv dan cloze_test test _spring2016.csv. Untuk versi 2018, file validasi dan pengujian perlu diganti namanya masing-masing menjadi cloze_test val _winter2018.csv dan menjadi cloze_test test _winter2018.csv. Pindahkan kedua file ini ke direktori manual.

  • Di-cache otomatis ( dokumentasi ): Ya

  • Struktur fitur :

FeaturesDict({
    'context': Text(shape=(), dtype=string),
    'endings': Sequence(Text(shape=(), dtype=string)),
    'label': int32,
})
  • Dokumentasi fitur :
Fitur Kelas Membentuk Dtype Keterangan
fiturDict
konteks Teks rangkaian
akhiran Urutan (Teks) (Tidak ada,) rangkaian
label Tensor int32
@inproceedings{sharma-etal-2018-tackling,
    title = "Tackling the Story Ending Biases in The Story Cloze Test",
    author = "Sharma, Rishi  and
      Allen, James  and
      Bakhshandeh, Omid  and
      Mostafazadeh, Nasrin",
    booktitle = "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)",
    month = jul,
    year = "2018",
    address = "Melbourne, Australia",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/P18-2119",
    doi = "10.18653/v1/P18-2119",
    pages = "752--757",
    abstract = "The Story Cloze Test (SCT) is a recent framework for evaluating story comprehension and script learning. There have been a variety of models tackling the SCT so far. Although the original goal behind the SCT was to require systems to perform deep language understanding and commonsense reasoning for successful narrative understanding, some recent models could perform significantly better than the initial baselines by leveraging human-authorship biases discovered in the SCT dataset. In order to shed some light on this issue, we have performed various data analysis and analyzed a variety of top performing models presented for this task. Given the statistics we have aggregated, we have designed a new crowdsourcing scheme that creates a new SCT dataset, which overcomes some of the biases. We benchmark a few models on the new dataset and show that the top-performing model on the original SCT dataset fails to keep up its performance. Our findings further signify the importance of benchmarking NLP systems on various evolving test sets.",
}

story_cloze/2016 (konfigurasi default)

  • Ukuran dataset : 1.15 MiB

  • Perpecahan :

Membelah Contoh
'test' 1.871
'validation' 1.871

story_cloze/2018

  • Ukuran dataset : 1015.04 KiB

  • Perpecahan :

Membelah Contoh
'test' 1.571
'validation' 1.571