драгоценный камень

  • Описание :

GEM — эталонная среда для генерации естественного языка с упором на его оценку, как с помощью человеческих аннотаций, так и с помощью автоматических метрик.

GEM нацелен на: (1) измерение прогресса NLG по 13 наборам данных, охватывающим множество задач NLG и языков. (2) обеспечить углубленный анализ данных и моделей, представленных в отчетах о данных и наборах задач. (3) разработать стандарты для оценки сгенерированного текста с использованием как автоматических, так и человеческих показателей.

Дополнительную информацию можно найти на https://gem-benchmark.com .

  • Домашняя страница : https://gem-benchmark.com

  • Исходный код : tfds.text.gem.Gem

  • Версии :

    • 1.0.0 : Начальная версия
    • 1.0.1 : Обновлен фильтр плохих ссылок для MLSum.
    • 1.1.0 (по умолчанию): выпуск наборов испытаний
  • Ключи под наблюдением (см . документ as_supervised ): None

  • Рисунок ( tfds.show_examples ): не поддерживается.

gem/common_gen (конфигурация по умолчанию)

  • Описание конфигурации : CommonGen — это задача генерации текста с ограничениями, связанная с эталонным набором данных, для явного тестирования машин на способность к генеративным рассуждениям на основе здравого смысла. Учитывая набор общих понятий; задача состоит в том, чтобы с помощью этих понятий составить связное предложение, описывающее повседневный сценарий.

  • Размер загрузки : 1.84 MiB

  • Размер набора данных : 16.84 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'challenge_test_scramble' 500
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 1497
'train' 67 389
'validation' 993
  • Структура функции :
FeaturesDict({
    'concept_set_id': tf.int32,
    'concepts': Sequence(tf.string),
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'target': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
concept_set_id Тензор tf.int32
понятия Последовательность (тензор) (Никто,) tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
  • Цитата :
@inproceedings{lin2020commongen,
  title = "CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning",
  author = "Lin, Bill Yuchen  and
    Zhou, Wangchunshu  and
    Shen, Ming  and
    Zhou, Pei  and
    Bhagavatula, Chandra  and
    Choi, Yejin  and
    Ren, Xiang",
  booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
  month = nov,
  year = "2020",
  address = "Online",
  publisher = "Association for Computational Linguistics",
  url = "https://www.aclweb.org/anthology/2020.findings-emnlp.165",
  pages = "1823--1840",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/cs_restaurants

  • Описание конфига : Задача — генерировать ответы в контексте (гипотетической) диалоговой системы, предоставляющей информацию о ресторанах. Ввод представляет собой базовый тип намерения/диалога и список слотов (атрибутов) и их значений. На выходе получается предложение на естественном языке.

  • Размер загрузки : 1.46 MiB

  • Размер набора данных : 2.71 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'challenge_test_scramble' 500
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 842
'train' 3569
'validation' 781
  • Структура функции :
FeaturesDict({
    'dialog_act': tf.string,
    'dialog_act_delexicalized': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'target': tf.string,
    'target_delexicalized': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
диалог_акт Тензор tf.string
dialog_act_delexicalized Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
target_delexicalized Тензор tf.string
  • Цитата :
@inproceedings{cs_restaurants,
  address = {Tokyo, Japan},
  title = {Neural {Generation} for {Czech}: {Data} and {Baselines} },
  shorttitle = {Neural {Generation} for {Czech} },
  url = {https://www.aclweb.org/anthology/W19-8670/},
  urldate = {2019-10-18},
  booktitle = {Proceedings of the 12th {International} {Conference} on {Natural} {Language} {Generation} ({INLG} 2019)},
  author = {Dušek, Ondřej and Jurčíček, Filip},
  month = oct,
  year = {2019},
  pages = {563--574}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/дротик

  • Описание конфигурации : DART — это большой структурированный корпус DAta Record to Text с открытым доменом, содержащий высококачественные аннотации предложений, где каждый вход представляет собой набор троек отношений между сущностями, следующих онтологии с древовидной структурой.

  • Размер загрузки : 28.01 MiB

  • Размер набора данных : 33.78 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 6959
'train' 62 659
'validation' 2768
  • Структура функции :
FeaturesDict({
    'dart_id': tf.int32,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'subtree_was_extended': tf.bool,
    'target': tf.string,
    'target_sources': Sequence(tf.string),
    'tripleset': Sequence(tf.string),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
dart_id Тензор tf.int32
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
subtree_was_extended Тензор tf.bool
цель Тензор tf.string
target_sources Последовательность (тензор) (Никто,) tf.string
тройной сет Последовательность (тензор) (Никто,) tf.string
  • Цитата :
@article{radev2020dart,
  title=Dart: Open-domain structured data record to text generation,
  author={Radev, Dragomir and Zhang, Rui and Rau, Amrit and Sivaprasad, Abhinand and Hsieh, Chiachun and Rajani, Nazneen Fatema and Tang, Xiangru and Vyas, Aadit and Verma, Neha and Krishna, Pranav and others},
  journal={arXiv preprint arXiv:2007.02871},
  year={2020}
}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/e2e_nlg

  • Описание конфигурации : набор данных E2E предназначен для задачи преобразования данных в текст с ограниченной областью — создание описаний/рекомендаций ресторана на основе до 8 различных атрибутов (название, район, ценовой диапазон и т. д.).

  • Размер загрузки : 13.99 MiB

  • Размер набора данных : 16.92 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'challenge_test_scramble' 500
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 4693
'train' 33 525
'validation' 4299
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'meaning_representation': tf.string,
    'references': Sequence(tf.string),
    'target': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
смысл_представление Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
  • Цитата :
@inproceedings{e2e_cleaned,
  address = {Tokyo, Japan},
  title = {Semantic {Noise} {Matters} for {Neural} {Natural} {Language} {Generation} },
  url = {https://www.aclweb.org/anthology/W19-8652/},
  booktitle = {Proceedings of the 12th {International} {Conference} on {Natural} {Language} {Generation} ({INLG} 2019)},
  author = {Dušek, Ondřej and Howcroft, David M and Rieser, Verena},
  year = {2019},
  pages = {421--426},
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/mlsum_de

  • Описание конфигурации : MLSum — это крупномасштабный многоязычный набор данных для суммирования. Он составлен из новостных онлайн-изданий, в этом разделе основное внимание уделяется немецкому языку.

  • Размер загрузки : 345.98 MiB

  • Размер набора данных : 963.60 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'challenge_test_covid' 5058
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 10 695
'train' 220 748
'validation' 11 392
  • Структура функции :
FeaturesDict({
    'date': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'target': tf.string,
    'text': tf.string,
    'title': tf.string,
    'topic': tf.string,
    'url': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
свидание Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
текст Тензор tf.string
заглавие Тензор tf.string
тема Тензор tf.string
URL Тензор tf.string
  • Цитата :
@inproceedings{scialom-etal-2020-mlsum,
    title = "{MLSUM}: The Multilingual Summarization Corpus",
    author = {Scialom, Thomas  and Dray, Paul-Alexis  and Lamprier, Sylvain  and Piwowarski, Benjamin  and Staiano, Jacopo},
    booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
    year = {2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/mlsum_es

  • Описание конфигурации : MLSum — это крупномасштабный многоязычный набор данных для суммирования. Он составлен из новостных онлайн-изданий, в этом разделе основное внимание уделяется испанскому языку.

  • Размер загрузки : 501.27 MiB

  • Размер набора данных : 1.29 GiB

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'challenge_test_covid' 1938
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 13 366
'train' 259 888
'validation' 9977
  • Структура функции :
FeaturesDict({
    'date': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'target': tf.string,
    'text': tf.string,
    'title': tf.string,
    'topic': tf.string,
    'url': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
свидание Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
текст Тензор tf.string
заглавие Тензор tf.string
тема Тензор tf.string
URL Тензор tf.string
  • Цитата :
@inproceedings{scialom-etal-2020-mlsum,
    title = "{MLSUM}: The Multilingual Summarization Corpus",
    author = {Scialom, Thomas  and Dray, Paul-Alexis  and Lamprier, Sylvain  and Piwowarski, Benjamin  and Staiano, Jacopo},
    booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
    year = {2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/schema_guided_dialog

  • Описание конфигурации : набор данных Schema-Guided Dialogue (SGD) содержит 18 000 многодоменных диалогов, ориентированных на задачи, между человеком и виртуальным помощником, которые охватывают 17 доменов — от банков и событий до медиа, календаря, путешествий и погоды.

  • Размер загрузки : 17.00 MiB

  • Размер набора данных : 201.19 MiB .

  • Автоматическое кэширование ( документация ): да (challenge_test_backtranslation, challenge_test_bfp02, challenge_test_bfp05, challenge_test_nopunc, challenge_test_scramble, challenge_train_sample, challenge_validation_sample, test, validation), только если shuffle_files=False (поезд)

  • Сплиты :

Расколоть Примеры
'challenge_test_backtranslation' 500
'challenge_test_bfp02' 500
'challenge_test_bfp05' 500
'challenge_test_nopunc' 500
'challenge_test_scramble' 500
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 10 000
'train' 164 982
'validation' 10 000
  • Структура функции :
FeaturesDict({
    'context': Sequence(tf.string),
    'dialog_acts': Sequence({
        'act': ClassLabel(shape=(), dtype=tf.int64, num_classes=18),
        'slot': tf.string,
        'values': Sequence(tf.string),
    }),
    'dialog_id': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'prompt': tf.string,
    'references': Sequence(tf.string),
    'service': tf.string,
    'target': tf.string,
    'turn_id': tf.int32,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
контекст Последовательность (тензор) (Никто,) tf.string
диалог_акты Последовательность
dialog_acts/акт Метка класса tf.int64
dialog_acts/слот Тензор tf.string
dialog_acts/значения Последовательность (тензор) (Никто,) tf.string
dialog_id Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
быстрый Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
оказание услуг Тензор tf.string
цель Тензор tf.string
поворот_идентификатор Тензор tf.int32
  • Цитата :
@article{rastogi2019towards,
  title={Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset},
  author={Rastogi, Abhinav and Zang, Xiaoxue and Sunkara, Srinivas and Gupta, Raghav and Khaitan, Pranav},
  journal={arXiv preprint arXiv:1909.05855},
  year={2019}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/тотто

  • Описание конфигурации : ToTTo — это задача преобразования таблицы в текст NLG. Задача заключается в следующем: для данной таблицы Википедии с именами строк, именами столбцов и ячейками таблицы с выделенным подмножеством ячеек сгенерируйте описание на естественном языке для выделенной части таблицы.

  • Размер загрузки : 180.75 MiB

  • Размер набора данных : 645.86 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'challenge_test_scramble' 500
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 7700
'train' 121 153
'validation' 7700
  • Структура функции :
FeaturesDict({
    'example_id': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'highlighted_cells': Sequence(Sequence(tf.int32)),
    'overlap_subset': tf.string,
    'references': Sequence(tf.string),
    'sentence_annotations': Sequence({
        'final_sentence': tf.string,
        'original_sentence': tf.string,
        'sentence_after_ambiguity': tf.string,
        'sentence_after_deletion': tf.string,
    }),
    'table': Sequence(Sequence({
        'column_span': tf.int32,
        'is_header': tf.bool,
        'row_span': tf.int32,
        'value': tf.string,
    })),
    'table_page_title': tf.string,
    'table_section_text': tf.string,
    'table_section_title': tf.string,
    'table_webpage_url': tf.string,
    'target': tf.string,
    'totto_id': tf.int32,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
example_id Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
выделенные_ячейки Последовательность (Последовательность (Тензор)) (Нет, Нет) tf.int32
перекрытие_подмножество Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
предложение_аннотации Последовательность
предложение_аннотации/окончательное_предложение Тензор tf.string
предложение_аннотации/оригинальное_предложение Тензор tf.string
предложение_аннотаций/предложение_после_двусмысленности Тензор tf.string
предложение_аннотации/предложение_после_удаления Тензор tf.string
стол Последовательность
таблица/column_span Тензор tf.int32
таблица/is_header Тензор tf.bool
таблица / row_span Тензор tf.int32
таблица/значение Тензор tf.string
table_page_title Тензор tf.string
table_section_text Тензор tf.string
table_section_title Тензор tf.string
table_webpage_url Тензор tf.string
цель Тензор tf.string
totto_id Тензор tf.int32
  • Цитата :
@inproceedings{parikh2020totto,
  title=ToTTo: A Controlled Table-To-Text Generation Dataset,
  author={Parikh, Ankur and Wang, Xuezhi and Gehrmann, Sebastian and Faruqui, Manaal and Dhingra, Bhuwan and Yang, Diyi and Das, Dipanjan},
  booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  pages={1173--1186},
  year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/web_nlg_en

  • Описание конфигурации : WebNLG — это двуязычный набор данных (английский, русский) параллельных наборов троек DBpedia и коротких текстов, охватывающих около 450 различных свойств DBpedia. Данные WebNLG изначально были созданы для содействия развитию вербализаторов RDF, способных генерировать короткие тексты и выполнять микропланирование.

  • Размер загрузки : 12.57 MiB

  • Размер набора данных : 19.91 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'challenge_test_numbers' 500
'challenge_test_scramble' 500
'challenge_train_sample' 502
'challenge_validation_sample' 499
'test' 1779
'train' 35 426
'validation' 1667
  • Структура функции :
FeaturesDict({
    'category': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'input': Sequence(tf.string),
    'references': Sequence(tf.string),
    'target': tf.string,
    'webnlg_id': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
категория Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
вход Последовательность (тензор) (Никто,) tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
webnlg_id Тензор tf.string
  • Цитата :
@inproceedings{gardent2017creating,
  author = "Gardent, Claire
    and Shimorina, Anastasia
    and Narayan, Shashi
    and Perez-Beltrachini, Laura",
  title = "Creating Training Corpora for NLG Micro-Planners",
  booktitle = "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year = "2017",
  publisher = "Association for Computational Linguistics",
  pages = "179--188",
  location = "Vancouver, Canada",
  doi = "10.18653/v1/P17-1017",
  url = "http://www.aclweb.org/anthology/P17-1017"
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

гем/web_nlg_ru

  • Описание конфигурации : WebNLG — это двуязычный набор данных (английский, русский) параллельных наборов троек DBpedia и коротких текстов, охватывающих около 450 различных свойств DBpedia. Данные WebNLG изначально были созданы для содействия развитию вербализаторов RDF, способных генерировать короткие тексты и выполнять микропланирование.

  • Размер загрузки : 7.49 MiB

  • Размер набора данных : 11.30 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'challenge_test_scramble' 500
'challenge_train_sample' 501
'challenge_validation_sample' 500
'test' 1102
'train' 14 630
'validation' 790
  • Структура функции :
FeaturesDict({
    'category': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'input': Sequence(tf.string),
    'references': Sequence(tf.string),
    'target': tf.string,
    'webnlg_id': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
категория Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
вход Последовательность (тензор) (Никто,) tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
webnlg_id Тензор tf.string
  • Цитата :
@inproceedings{gardent2017creating,
  author = "Gardent, Claire
    and Shimorina, Anastasia
    and Narayan, Shashi
    and Perez-Beltrachini, Laura",
  title = "Creating Training Corpora for NLG Micro-Planners",
  booktitle = "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year = "2017",
  publisher = "Association for Computational Linguistics",
  pages = "179--188",
  location = "Vancouver, Canada",
  doi = "10.18653/v1/P17-1017",
  url = "http://www.aclweb.org/anthology/P17-1017"
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_auto_asset_turk

  • Описание конфигурации : WikiAuto предоставляет набор выровненных предложений из английской Википедии и Simple English Wikipedia в качестве ресурса для обучения систем упрощения предложений. ASSET и TURK — это высококачественные наборы данных упрощения, используемые для тестирования.

  • Размер загрузки : 121.01 MiB

  • Размер набора данных : 202.40 MiB .

  • Auto-cached ( documentation ): Yes (challenge_test_asset_backtranslation, challenge_test_asset_bfp02, challenge_test_asset_bfp05, challenge_test_asset_nopunc, challenge_test_turk_backtranslation, challenge_test_turk_bfp02, challenge_test_turk_bfp05, challenge_test_turk_nopunc, challenge_train_sample, challenge_validation_sample, test_asset, test_turk, validation), Only when shuffle_files=False (train)

  • Сплиты :

Расколоть Примеры
'challenge_test_asset_backtranslation' 359
'challenge_test_asset_bfp02' 359
'challenge_test_asset_bfp05' 359
'challenge_test_asset_nopunc' 359
'challenge_test_turk_backtranslation' 359
'challenge_test_turk_bfp02' 359
'challenge_test_turk_bfp05' 359
'challenge_test_turk_nopunc' 359
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test_asset' 359
'test_turk' 359
'train' 483 801
'validation' 20 000
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'target': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
цель Тензор tf.string
  • Цитата :
@inproceedings{jiang-etal-2020-neural,
    title = "Neural {CRF} Model for Sentence Alignment in Text Simplification",
    author = "Jiang, Chao  and
      Maddela, Mounica  and
      Lan, Wuwei  and
      Zhong, Yang  and
      Xu, Wei",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-main.709",
    doi = "10.18653/v1/2020.acl-main.709",
    pages = "7943--7960",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/сумма

  • Описание конфигурации : набор данных предназначен для задачи абстрактного обобщения в его крайней форме, речь идет об обобщении документа в одном предложении.

  • Размер загрузки : 246.31 MiB

  • Размер набора данных : 78.89 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'challenge_test_backtranslation' 500
'challenge_test_bfp_02' 500
'challenge_test_bfp_05' 500
'challenge_test_covid' 401
'challenge_test_nopunc' 500
'challenge_train_sample' 500
'challenge_validation_sample' 500
'test' 1166
'train' 23 206
'validation' 1117
  • Структура функции :
FeaturesDict({
    'document': tf.string,
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'target': tf.string,
    'xsum_id': tf.string,
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
документ Тензор tf.string
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
цель Тензор tf.string
xsum_id Тензор tf.string
  • Цитата :
@inproceedings{Narayan2018dont,
  author = "Shashi Narayan and Shay B. Cohen and Mirella Lapata",
  title = "Don't Give Me the Details, Just the Summary! {T}opic-Aware Convolutional Neural Networks for Extreme Summarization",
  booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing ",
  year = "2018",
  address = "Brussels, Belgium",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_arabic_ar

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 56.25 MiB .

  • Размер набора данных : 291.42 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 5841
'train' 20 441
'validation' 2919
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'ar': Text(shape=(), dtype=tf.string),
        'en': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'ar': Text(shape=(), dtype=tf.string),
        'en': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ar Текст tf.string
source_aligned/ru Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ar Текст tf.string
target_aligned/ru Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_chinese_zh

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 31.38 MiB

  • Размер набора данных : 122.06 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 3775
'train' 13 211
'validation' 1886
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'zh': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'zh': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/ж Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/ж Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_czech_cs

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 13.84 MiB

  • Размер набора данных : 58.05 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 1438
'train' 5033
'validation' 718
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'cs': Text(shape=(), dtype=tf.string),
        'en': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'cs': Text(shape=(), dtype=tf.string),
        'en': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/cs Текст tf.string
source_aligned/ru Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/cs Текст tf.string
target_aligned/ru Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_dutch_nl

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 53.88 MiB

  • Размер набора данных : 237.97 MiB .

  • Автоматическое кэширование ( документация ): да (тест, проверка), только если shuffle_files=False (поезд)

  • Сплиты :

Расколоть Примеры
'test' 6248
'train' 21 866
'validation' 3123
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'nl': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'nl': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/nl Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/nl Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_english_en

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 112.56 MiB

  • Размер набора данных : 657.51 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 28 614
'train' 99 020
'validation' 13 823
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

жемчужина/wiki_lingua_french_fr

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 113.26 MiB

  • Размер набора данных : 522.28 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 12 731
'train' 44 556
'validation' 6364
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'fr': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'fr': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/fr Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/fr Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_german_de

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 102.65 MiB

  • Размер набора данных : 452.46 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 11 669
'train' 40 839
'validation' 5833
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'de': Text(shape=(), dtype=tf.string),
        'en': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'de': Text(shape=(), dtype=tf.string),
        'en': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/de Текст tf.string
source_aligned/ru Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/de Текст tf.string
target_aligned/ru Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_hindi_hi

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 20.07 MiB

  • Размер набора данных : 138.06 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 1984
'train' 6942
'validation' 991
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'hi': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'hi': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/привет Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/привет Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_indonesian_id

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 80.08 MiB

  • Размер набора данных : 370.63 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 9497
'train' 33 237
'validation' 4747
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'id': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'id': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/идентификатор Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/идентификатор Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_italian_it

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 84.80 MiB

  • Размер набора данных : 374.40 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 10 189
'train' 35 661
'validation' 5093
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'it': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'it': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/это Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/это Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_japanese_ja

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 21.75 MiB .

  • Размер набора данных : 103.19 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 2530
'train' 8853
'validation' 1264
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'ja': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'ja': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/ja Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/ja Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_korean_ko

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 22.26 MiB

  • Размер набора данных : 102.35 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 2436
'train' 8 524
'validation' 1216
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'ko': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'ko': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/ko Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/ko Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_portuguese_pt

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 131.17 MiB

  • Размер набора данных : 570.46 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 16 331
'train' 57 159
'validation' 8165
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'pt': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'pt': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/pt Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/pt Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_russian_ru

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 101.36 MiB

  • Размер набора данных : 564.69 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 10 580
'train' 37 028
'validation' 5288
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'ru': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'ru': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/ru Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/ru Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_spanish_es

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 189.06 MiB

  • Размер набора данных : 849.75 MiB .

  • Автоматическое кэширование ( документация ): Нет

  • Сплиты :

Расколоть Примеры
'test' 22 632
'train' 79 212
'validation' 11 316
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'es': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'es': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/es Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/es Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_thai_th

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 28.60 MiB

  • Размер набора данных : 193.77 MiB .

  • Автоматическое кэширование ( документация ): да (тест, проверка), только если shuffle_files=False (поезд)

  • Сплиты :

Расколоть Примеры
'test' 2950
'train' 10 325
'validation' 1475
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'th': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'th': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/th Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/th Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

драгоценный камень/wiki_lingua_turkish_tr

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 6.73 MiB

  • Размер набора данных : 30.75 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 900
'train' 3148
'validation' 449
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'tr': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'tr': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/tr Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/tr Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

жемчужина/wiki_lingua_vietnamese_vi

  • Описание конфигурации : Wikilingua — это крупномасштабный многоязычный набор данных для оценки межъязыковых систем абстрактного обобщения.

  • Размер загрузки : 36.27 MiB

  • Размер набора данных : 179.77 MiB .

  • Автоматическое кэширование ( документация ): Да

  • Сплиты :

Расколоть Примеры
'test' 3917
'train' 13 707
'validation' 1957
  • Структура функции :
FeaturesDict({
    'gem_id': tf.string,
    'gem_parent_id': tf.string,
    'references': Sequence(tf.string),
    'source': tf.string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'vi': Text(shape=(), dtype=tf.string),
    }),
    'target': tf.string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=tf.string),
        'vi': Text(shape=(), dtype=tf.string),
    }),
})
  • Документация по функциям :
Особенность Учебный класс Форма Dтип Описание
ОсобенностиDict
gem_id Тензор tf.string
gem_parent_id Тензор tf.string
использованная литература Последовательность (тензор) (Никто,) tf.string
источник Тензор tf.string
source_aligned Перевод
source_aligned/ru Текст tf.string
source_aligned/vi Текст tf.string
цель Тензор tf.string
target_aligned Перевод
target_aligned/ru Текст tf.string
target_aligned/vi Текст tf.string
  • Цитата :
@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."