פְּנִינָה

• תיאור :

GEM היא סביבת אמת מידה ליצירת שפה טבעית עם התמקדות בהערכתה, הן באמצעות הערות אנושיות והן באמצעות מדדים אוטומטיים.

GEM שואפת: (1) למדוד התקדמות NLG על פני 13 מערכי נתונים המשתרעים על משימות ושפות רבות של NLG. (2) לספק ניתוח מעמיק של נתונים ומודלים המוצגים באמצעות הצהרות נתונים וקבוצות אתגרים. (3) לפתח סטנדרטים להערכת טקסט שנוצר תוך שימוש במדדים אוטומטיים ואנושיים כאחד.

מידע נוסף ניתן למצוא בכתובת https://gem-benchmark.com .

• תיעוד נוסף : חקור על ניירות עם קוד

• דף הבית : https://gem-benchmark.com

• קוד מקור : tfds.text.gem.Gem

• גרסאות :

  • 1.0.0 : גרסה ראשונית
  • 1.0.1 : עדכון מסנן קישורים לא טובים עבור MLSum
  • 1.1.0 (ברירת מחדל): שחרור ערכות האתגר

• מפתחות בפיקוח (ראה as_supervised doc ): None

• איור ( tfds.show_examples ): לא נתמך.

gem/common_gen (תצורת ברירת מחדל)

• תיאור תצורה : CommonGen היא משימת יצירת טקסט מוגבלת, המשויכת למערך נתונים של benchmark, כדי לבדוק במפורש מכונות ליכולת של היגיון בריא. נתון קבוצה של מושגים נפוצים; המשימה היא ליצור משפט קוהרנטי המתאר תרחיש יומיומי באמצעות מושגים אלה.

• גודל הורדה : 1.84 MiB

• גודל ערכת נתונים : 16.84 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	1,497
'train'	67,389
'validation'	993

• מבנה תכונה :

FeaturesDict({
    'concept_set_id': int32,
    'concepts': Sequence(string),
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
concept_set_id	מוֹתֵחַ		int32
מושגים	רצף (טנזור)	(אף אחד,)	חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{lin2020commongen,
  title = "CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning",
  author = "Lin, Bill Yuchen  and
    Zhou, Wangchunshu  and
    Shen, Ming  and
    Zhou, Pei  and
    Bhagavatula, Chandra  and
    Choi, Yejin  and
    Ren, Xiang",
  booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
  month = nov,
  year = "2020",
  address = "Online",
  publisher = "Association for Computational Linguistics",
  url = "https://www.aclweb.org/anthology/2020.findings-emnlp.165",
  pages = "1823--1840",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/cs_restaurants

• תיאור תצורה : המשימה מייצרת תגובות בהקשר של מערכת דיאלוג (היפותטית) המספקת מידע על מסעדות. הקלט הוא סוג כוונה/דיאלוג בסיסי ורשימת משבצות (תכונות) והערכים שלהם. הפלט הוא משפט בשפה טבעית.

• גודל הורדה : 1.46 MiB

• גודל ערכת נתונים : 2.71 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	842
'train'	3,569
'validation'	781

• מבנה תכונה :

FeaturesDict({
    'dialog_act': string,
    'dialog_act_delexicalized': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'target_delexicalized': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
דיאלוג_אקט	מוֹתֵחַ		חוּט
דיאלוג_אקט_מחוקק	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט
target_delexicalized	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{cs_restaurants,
  address = {Tokyo, Japan},
  title = {Neural {Generation} for {Czech}: {Data} and {Baselines} },
  shorttitle = {Neural {Generation} for {Czech} },
  url = {https://www.aclweb.org/anthology/W19-8670/},
  urldate = {2019-10-18},
  booktitle = {Proceedings of the 12th {International} {Conference} on {Natural} {Language} {Generation} ({INLG} 2019)},
  author = {Dušek, Ondřej and Jurčíček, Filip},
  month = oct,
  year = {2019},
  pages = {563--574}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

אבן חן/חץ

• תיאור תצורה : DART הוא קורפוס גדול ופתוח מובנה של רשומות DAta לטקסט עם הערות משפטיות באיכות גבוהה כאשר כל קלט הוא קבוצה של משולשים ביחסי ישות בעקבות אונטולוגיה מובנית עצים.

• גודל הורדה : 28.01 MiB

• גודל ערכת נתונים: 33.78 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	6,959
'train'	62,659
'validation'	2,768

• מבנה תכונה :

FeaturesDict({
    'dart_id': int32,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'subtree_was_extended': bool,
    'target': string,
    'target_sources': Sequence(string),
    'tripleset': Sequence(string),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
dart_id	מוֹתֵחַ		int32
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
עץ המשנה_הורחב	מוֹתֵחַ		bool
יַעַד	מוֹתֵחַ		חוּט
target_sources	רצף (טנזור)	(אף אחד,)	חוּט
ערכה משולשת	רצף (טנזור)	(אף אחד,)	חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@article{radev2020dart,
  title=Dart: Open-domain structured data record to text generation,
  author={Radev, Dragomir and Zhang, Rui and Rau, Amrit and Sivaprasad, Abhinand and Hsieh, Chiachun and Rajani, Nazneen Fatema and Tang, Xiangru and Vyas, Aadit and Verma, Neha and Krishna, Pranav and others},
  journal={arXiv preprint arXiv:2007.02871},
  year={2020}
}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/e2e_nlg

• תיאור תצורה : מערך הנתונים של E2E מיועד למשימת נתונים לטקסט של תחום מוגבל - יצירת תיאורים/המלצות של מסעדות המבוססות על עד 8 מאפיינים שונים (שם, אזור, טווח מחירים וכו')

• גודל הורדה : 13.99 MiB

• גודל ערכת נתונים: 16.92 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	4,693
'train'	33,525
'validation'	4,299

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'meaning_representation': string,
    'references': Sequence(string),
    'target': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
ייצוג_משמעות	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{e2e_cleaned,
  address = {Tokyo, Japan},
  title = {Semantic {Noise} {Matters} for {Neural} {Natural} {Language} {Generation} },
  url = {https://www.aclweb.org/anthology/W19-8652/},
  booktitle = {Proceedings of the 12th {International} {Conference} on {Natural} {Language} {Generation} ({INLG} 2019)},
  author = {Dušek, Ondřej and Howcroft, David M and Rieser, Verena},
  year = {2019},
  pages = {421--426},
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/mlsum_de

• תיאור תצורה : MLSum הוא מערך סיכום רב לשוני בקנה מידה גדול. הוא בנוי מכלי חדשות מקוונים, הפיצול הזה מתמקד בגרמנית.

• גודל הורדה : 345.98 MiB

• גודל ערכת נתונים: 963.60 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_covid'	5,058
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	10,695
'train'	220,748
'validation'	11,392

• מבנה תכונה :

FeaturesDict({
    'date': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'text': string,
    'title': string,
    'topic': string,
    'url': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
תַאֲרִיך	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט
טֶקסט	מוֹתֵחַ		חוּט
כותרת	מוֹתֵחַ		חוּט
נוֹשֵׂא	מוֹתֵחַ		חוּט
כתובת אתר	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{scialom-etal-2020-mlsum,
    title = "{MLSUM}: The Multilingual Summarization Corpus",
    author = {Scialom, Thomas  and Dray, Paul-Alexis  and Lamprier, Sylvain  and Piwowarski, Benjamin  and Staiano, Jacopo},
    booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
    year = {2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/mlsum_es

• תיאור תצורה : MLSum הוא מערך סיכום רב לשוני בקנה מידה גדול. הוא בנוי מכלי חדשות מקוונים, הפיצול הזה מתמקד בספרדית.

• גודל הורדה : 501.27 MiB

• גודל מערך נתונים : 1.29 GiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_covid'	1,938
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	13,366
'train'	259,888
'validation'	9,977

• מבנה תכונה :

FeaturesDict({
    'date': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'text': string,
    'title': string,
    'topic': string,
    'url': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
תַאֲרִיך	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט
טֶקסט	מוֹתֵחַ		חוּט
כותרת	מוֹתֵחַ		חוּט
נוֹשֵׂא	מוֹתֵחַ		חוּט
כתובת אתר	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{scialom-etal-2020-mlsum,
    title = "{MLSUM}: The Multilingual Summarization Corpus",
    author = {Scialom, Thomas  and Dray, Paul-Alexis  and Lamprier, Sylvain  and Piwowarski, Benjamin  and Staiano, Jacopo},
    booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
    year = {2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/schema_guided_dialog

• תיאור תצורה : מערך הנתונים של Schema-Guided Dialogue (SGD) מכיל דיאלוגים מוכווני משימות של 18K מרובי דומיינים בין אדם לעוזר וירטואלי, המכסה 17 תחומים, החל מבנקים ואירועים ועד מדיה, לוח שנה, נסיעות ומזג אוויר.

• גודל הורדה : 17.00 MiB

• גודל ערכת נתונים: 201.19 MiB

• מטמון אוטומטי ( תיעוד ): כן (challenge_test_backtranslation, challenge_test_bfp02, challenge_test_bfp05, challenge_test_nopunc, challenge_test_scramble, challenge_train_sample, challenge_validation_sample, test, validation), רק כאשר shuffle_files=False (רכב)

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_backtranslation'	500
'challenge_test_bfp02'	500
'challenge_test_bfp05'	500
'challenge_test_nopunc'	500
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	10,000
'train'	164,982
'validation'	10,000

• מבנה תכונה :

FeaturesDict({
    'context': Sequence(string),
    'dialog_acts': Sequence({
        'act': ClassLabel(shape=(), dtype=int64, num_classes=18),
        'slot': string,
        'values': Sequence(string),
    }),
    'dialog_id': string,
    'gem_id': string,
    'gem_parent_id': string,
    'prompt': string,
    'references': Sequence(string),
    'service': string,
    'target': string,
    'turn_id': int32,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
הֶקשֵׁר	רצף (טנזור)	(אף אחד,)	חוּט
דיאלוג_פעולות	סדר פעולות
dialog_acts/act	ClassLabel		int64
dialog_acts/חריץ	מוֹתֵחַ		חוּט
דיאלוג_פעולות/ערכים	רצף (טנזור)	(אף אחד,)	חוּט
dialog_id	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
מיידי	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
שֵׁרוּת	מוֹתֵחַ		חוּט
יַעַד	מוֹתֵחַ		חוּט
turn_id	מוֹתֵחַ		int32

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@article{rastogi2019towards,
  title={Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset},
  author={Rastogi, Abhinav and Zang, Xiaoxue and Sunkara, Srinivas and Gupta, Raghav and Khaitan, Pranav},
  journal={arXiv preprint arXiv:1909.05855},
  year={2019}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

אבן חן/טוטו

• תיאור תצורה : ToTTo היא משימת NLG של Table-to-Text. המשימה היא כדלקמן: בהינתן טבלת ויקיפדיה עם שמות שורות, שמות עמודות ותאי טבלה, עם קבוצת משנה של תאים מודגשת, צור תיאור שפה טבעית עבור החלק המודגש של הטבלה.

• גודל הורדה : 180.75 MiB

• גודל ערכת נתונים: 645.86 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	7,700
'train'	121,153
'validation'	7,700

• מבנה תכונה :

FeaturesDict({
    'example_id': string,
    'gem_id': string,
    'gem_parent_id': string,
    'highlighted_cells': Sequence(Sequence(int32)),
    'overlap_subset': string,
    'references': Sequence(string),
    'sentence_annotations': Sequence({
        'final_sentence': string,
        'original_sentence': string,
        'sentence_after_ambiguity': string,
        'sentence_after_deletion': string,
    }),
    'table': Sequence(Sequence({
        'column_span': int32,
        'is_header': bool,
        'row_span': int32,
        'value': string,
    })),
    'table_page_title': string,
    'table_section_text': string,
    'table_section_title': string,
    'table_webpage_url': string,
    'target': string,
    'totto_id': int32,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
example_id	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
תאים_מודגשים	Sequence(Sequence(Tensor))	(אין, אין)	int32
overlap_subset	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
הערות_משפט	סדר פעולות
הערות_משפט/משפט_סופי	מוֹתֵחַ		חוּט
הערות_משפט/משפט_מקורי	מוֹתֵחַ		חוּט
הערות_משפט/משפט_אחרי_עמימות	מוֹתֵחַ		חוּט
הערות_משפט/משפט_אחרי_מחיקה	מוֹתֵחַ		חוּט
שולחן	סדר פעולות
table/column_span	מוֹתֵחַ		int32
table/is_header	מוֹתֵחַ		bool
table/row_span	מוֹתֵחַ		int32
טבלה/ערך	מוֹתֵחַ		חוּט
table_page_title	מוֹתֵחַ		חוּט
table_section_text	מוֹתֵחַ		חוּט
table_section_title	מוֹתֵחַ		חוּט
table_webpage_url	מוֹתֵחַ		חוּט
יַעַד	מוֹתֵחַ		חוּט
totto_id	מוֹתֵחַ		int32

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{parikh2020totto,
  title=ToTTo: A Controlled Table-To-Text Generation Dataset,
  author={Parikh, Ankur and Wang, Xuezhi and Gehrmann, Sebastian and Faruqui, Manaal and Dhingra, Bhuwan and Yang, Diyi and Das, Dipanjan},
  booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  pages={1173--1186},
  year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/web_nlg_en

• תיאור תצורה : WebNLG הוא מערך נתונים דו-לשוני (אנגלית, רוסית) של ערכות משולשות מקבילות של DBpedia וטקסטים קצרים המכסים כ-450 מאפייני DBpedia שונים. נתוני WebNLG נוצרו במקור כדי לקדם את הפיתוח של מילולי RDF המסוגלים ליצור טקסט קצר ולטפל בתכנון מיקרו.

• גודל הורדה : 12.57 MiB

• גודל מערך נתונים : 19.91 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_numbers'	500
'challenge_test_scramble'	500
'challenge_train_sample'	502
'challenge_validation_sample'	499
'test'	1,779
'train'	35,426
'validation'	1,667

• מבנה תכונה :

FeaturesDict({
    'category': string,
    'gem_id': string,
    'gem_parent_id': string,
    'input': Sequence(string),
    'references': Sequence(string),
    'target': string,
    'webnlg_id': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
קטגוריה	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
קֶלֶט	רצף (טנזור)	(אף אחד,)	חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט
webnlg_id	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{gardent2017creating,
  author = "Gardent, Claire
    and Shimorina, Anastasia
    and Narayan, Shashi
    and Perez-Beltrachini, Laura",
  title = "Creating Training Corpora for NLG Micro-Planners",
  booktitle = "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year = "2017",
  publisher = "Association for Computational Linguistics",
  pages = "179--188",
  location = "Vancouver, Canada",
  doi = "10.18653/v1/P17-1017",
  url = "http://www.aclweb.org/anthology/P17-1017"
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/web_nlg_ru

• תיאור תצורה : WebNLG הוא מערך נתונים דו-לשוני (אנגלית, רוסית) של ערכות משולשות מקבילות של DBpedia וטקסטים קצרים המכסים כ-450 מאפייני DBpedia שונים. נתוני WebNLG נוצרו במקור כדי לקדם את הפיתוח של מילולי RDF המסוגלים ליצור טקסט קצר ולטפל בתכנון מיקרו.

• גודל הורדה : 7.49 MiB

• גודל ערכת נתונים : 11.30 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_scramble'	500
'challenge_train_sample'	501
'challenge_validation_sample'	500
'test'	1,102
'train'	14,630
'validation'	790

• מבנה תכונה :

FeaturesDict({
    'category': string,
    'gem_id': string,
    'gem_parent_id': string,
    'input': Sequence(string),
    'references': Sequence(string),
    'target': string,
    'webnlg_id': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
קטגוריה	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
קֶלֶט	רצף (טנזור)	(אף אחד,)	חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט
webnlg_id	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{gardent2017creating,
  author = "Gardent, Claire
    and Shimorina, Anastasia
    and Narayan, Shashi
    and Perez-Beltrachini, Laura",
  title = "Creating Training Corpora for NLG Micro-Planners",
  booktitle = "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year = "2017",
  publisher = "Association for Computational Linguistics",
  pages = "179--188",
  location = "Vancouver, Canada",
  doi = "10.18653/v1/P17-1017",
  url = "http://www.aclweb.org/anthology/P17-1017"
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_auto_asset_turk

• תיאור תצורה : WikiAuto מספק קבוצה של משפטים מיושרים מויקיפדיה האנגלית ומוויקיפדיה האנגלית הפשוטה כמשאב לאימון מערכות פישוט משפטים. ASSET ו-TURK הם מערכי נתונים לפישוט באיכות גבוהה המשמשים לבדיקה.

• גודל הורדה : 121.01 MiB

• גודל מערך נתונים : 202.40 MiB

• Auto-cached ( documentation ): Yes (challenge_test_asset_backtranslation, challenge_test_asset_bfp02, challenge_test_asset_bfp05, challenge_test_asset_nopunc, challenge_test_turk_backtranslation, challenge_test_turk_bfp02, challenge_test_turk_bfp05, challenge_test_turk_nopunc, challenge_train_sample, challenge_validation_sample, test_asset, test_turk, validation), Only when shuffle_files=False (train)

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_asset_backtranslation'	359
'challenge_test_asset_bfp02'	359
'challenge_test_asset_bfp05'	359
'challenge_test_asset_nopunc'	359
'challenge_test_turk_backtranslation'	359
'challenge_test_turk_bfp02'	359
'challenge_test_turk_bfp05'	359
'challenge_test_turk_nopunc'	359
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test_asset'	359
'test_turk'	359
'train'	483,801
'validation'	20,000

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'target': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
יַעַד	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{jiang-etal-2020-neural,
    title = "Neural {CRF} Model for Sentence Alignment in Text Simplification",
    author = "Jiang, Chao  and
      Maddela, Mounica  and
      Lan, Wuwei  and
      Zhong, Yang  and
      Xu, Wei",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-main.709",
    doi = "10.18653/v1/2020.acl-main.709",
    pages = "7943--7960",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/xsum

• תיאור תצורה : מערך הנתונים נועד למשימה של סיכום מופשט בצורתו הקיצונית, מדובר בסיכום מסמך במשפט בודד.

• גודל הורדה : 246.31 MiB

• גודל ערכת נתונים: 78.89 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'challenge_test_backtranslation'	500
'challenge_test_bfp_02'	500
'challenge_test_bfp_05'	500
'challenge_test_covid'	401
'challenge_test_nopunc'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	1,166
'train'	23,206
'validation'	1,117

• מבנה תכונה :

FeaturesDict({
    'document': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'xsum_id': string,
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
מסמך	מוֹתֵחַ		חוּט
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
יַעַד	מוֹתֵחַ		חוּט
xsum_id	מוֹתֵחַ		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{Narayan2018dont,
  author = "Shashi Narayan and Shay B. Cohen and Mirella Lapata",
  title = "Don't Give Me the Details, Just the Summary! {T}opic-Aware Convolutional Neural Networks for Extreme Summarization",
  booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing ",
  year = "2018",
  address = "Brussels, Belgium",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_arabic_ar

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 56.25 MiB

• גודל ערכת נתונים: 291.42 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	5,841
'train'	20,441
'validation'	2,919

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'ar': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'ar': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/ar	טֶקסט		חוּט
source_aligned/he	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/ar	טֶקסט		חוּט
target_aligned/iw	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_chinese_zh

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 31.38 MiB

• גודל ערכת נתונים: 122.06 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	3,775
'train'	13,211
'validation'	1,886

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'zh': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'zh': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/zh	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/zh	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_czech_cs

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 13.84 MiB

• גודל ערכת נתונים: 58.05 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	1,438
'train'	5,033
'validation'	718

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'cs': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'cs': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/cs	טֶקסט		חוּט
source_aligned/he	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/cs	טֶקסט		חוּט
target_aligned/iw	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_dutch_nl

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 53.88 MiB

• גודל מערך נתונים : 237.97 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן (בדיקה, אימות), רק כאשר shuffle_files=False (רכבת)

• פיצולים :

לְפַצֵל	דוגמאות
'test'	6,248
'train'	21,866
'validation'	3,123

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'nl': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'nl': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/nl	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/nl	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_english_en

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 112.56 MiB

• גודל מערך נתונים : 657.51 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	28,614
'train'	99,020
'validation'	13,823

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_french_fr

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 113.26 MiB

• גודל מערך נתונים : 522.28 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	12,731
'train'	44,556
'validation'	6,364

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'fr': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'fr': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/fr	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/fr	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_german_de

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 102.65 MiB

• גודל ערכת נתונים: 452.46 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	11,669
'train'	40,839
'validation'	5,833

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'de': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'de': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/de	טֶקסט		חוּט
source_aligned/he	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/de	טֶקסט		חוּט
target_aligned/iw	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_hindi_hi

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 20.07 MiB

• גודל ערכת נתונים: 138.06 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	1,984
'train'	6,942
'validation'	991

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'hi': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'hi': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/hi	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/hi	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_indonesian_id

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 80.08 MiB

• גודל ערכת נתונים: 370.63 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	9,497
'train'	33,237
'validation'	4,747

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'id': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'id': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/id	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/id	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_italian_it

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 84.80 MiB

• גודל ערכת נתונים: 374.40 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	10,189
'train'	35,661
'validation'	5,093

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'it': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'it': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/it	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/it	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_japanese_ja

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 21.75 MiB

• גודל מערך נתונים : 103.19 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	2,530
'train'	8,853
'validation'	1,264

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ja': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ja': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/ja	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/ja	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_korean_ko

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 22.26 MiB

• גודל ערכת נתונים: 102.35 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	2,436
'train'	8,524
'validation'	1,216

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ko': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ko': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/ko	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/ko	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_portuguese_pt

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 131.17 MiB

• גודל ערכת נתונים: 570.46 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	16,331
'train'	57,159
'validation'	8,165

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'pt': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'pt': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/pt	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/pt	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_russian_ru

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 101.36 MiB

• גודל ערכת נתונים: 564.69 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	10,580
'train'	37,028
'validation'	5,288

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ru': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ru': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/ru	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/ru	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_spanish_es

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 189.06 MiB

• גודל מערך נתונים : 849.75 MiB

• שמירה אוטומטית במטמון ( תיעוד ): לא

• פיצולים :

לְפַצֵל	דוגמאות
'test'	22,632
'train'	79,212
'validation'	11,316

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'es': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'es': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/es	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/es	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_thai_th

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 28.60 MiB

• גודל ערכת נתונים: 193.77 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן (בדיקה, אימות), רק כאשר shuffle_files=False (רכבת)

• פיצולים :

לְפַצֵל	דוגמאות
'test'	2,950
'train'	10,325
'validation'	1,475

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'th': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'th': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/th	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/th	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_turkish_tr

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 6.73 MiB

• גודל ערכת נתונים : 30.75 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	900
'train'	3,148
'validation'	449

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'tr': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'tr': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/tr	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/tr	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_vietnamese_vi

• תיאור תצורה : Wikilingua הוא מערך נתונים רב לשוני בקנה מידה גדול להערכת מערכות סיכום מופשט חוצה לשוניות.

• גודל הורדה : 36.27 MiB

• גודל ערכת נתונים: 179.77 MiB

• שמור אוטומטי במטמון ( תיעוד ): כן

• פיצולים :

לְפַצֵל	דוגמאות
'test'	3,917
'train'	13,707
'validation'	1,957

• מבנה תכונה :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'vi': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'vi': Text(shape=(), dtype=string),
    }),
})

• תיעוד תכונה :

תכונה	מעמד	צוּרָה	Dtype	תיאור
	FeaturesDict
gem_id	מוֹתֵחַ		חוּט
gem_parent_id	מוֹתֵחַ		חוּט
הפניות	רצף (טנזור)	(אף אחד,)	חוּט
מָקוֹר	מוֹתֵחַ		חוּט
source_aligned	תִרגוּם
source_aligned/he	טֶקסט		חוּט
source_aligned/vi	טֶקסט		חוּט
יַעַד	מוֹתֵחַ		חוּט
target_aligned	תִרגוּם
target_aligned/iw	טֶקסט		חוּט
target_aligned/vi	טֶקסט		חוּט

• דוגמאות ( tfds.as_dataframe ):

• ציטוט :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."