보석

• 설명 :

GEM 은 인간의 주석과 자동화된 메트릭을 통한 평가에 중점을 둔 자연어 생성을 위한 벤치마크 환경입니다.

GEM의 목표는 다음과 같습니다. (1) 많은 NLG 작업 및 언어에 걸친 13개의 데이터 세트에서 NLG 진행 상황을 측정합니다. (2) 데이터 진술 및 챌린지 세트를 통해 제시된 데이터 및 모델에 대한 심층 분석을 제공합니다. (3) 자동 및 인간 메트릭을 모두 사용하여 생성된 텍스트를 평가하기 위한 표준을 개발합니다.

자세한 내용은 https://gem-benchmark.com 에서 확인할 수 있습니다.

• 추가 문서 : 코드가 있는 논문에서 탐색

• 홈페이지 : https://gem-benchmark.com

• 소스 코드 : tfds.text.gem.Gem

• 버전 :

  • 1.0.0 : 초기 버전
  • 1.0.1 : MLSum에 대한 잘못된 링크 필터 업데이트
  • 1.1.0 (기본): 챌린지 세트 출시

• 감독된 키 ( as_supervised 문서 참조): None

• 그림 ( tfds.show_examples ): 지원되지 않습니다.

gem/common_gen(기본 구성)

• 구성 설명 : CommonGen은 생성적 상식 추론 기능에 대해 기계를 명시적으로 테스트하기 위해 벤치마크 데이터 세트와 연결된 제한된 텍스트 생성 작업입니다. 일련의 일반적인 개념이 주어지면; 작업은 이러한 개념을 사용하여 일상적인 시나리오를 설명하는 일관된 문장을 생성하는 것입니다.

• 다운로드 크기 : 1.84 MiB

• 데이터 세트 크기 : 16.84 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	1,497
'train'	67,389
'validation'	993

• 기능 구조 :

FeaturesDict({
    'concept_set_id': int32,
    'concepts': Sequence(string),
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
concept_set_id	텐서		int32
개념	시퀀스(텐서)	(없음,)	끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{lin2020commongen,
  title = "CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning",
  author = "Lin, Bill Yuchen  and
    Zhou, Wangchunshu  and
    Shen, Ming  and
    Zhou, Pei  and
    Bhagavatula, Chandra  and
    Choi, Yejin  and
    Ren, Xiang",
  booktitle = "Findings of the Association for Computational Linguistics: EMNLP 2020",
  month = nov,
  year = "2020",
  address = "Online",
  publisher = "Association for Computational Linguistics",
  url = "https://www.aclweb.org/anthology/2020.findings-emnlp.165",
  pages = "1823--1840",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/cs_레스토랑

• 구성 설명 : 작업은 레스토랑에 대한 정보를 제공하는 (가상) 대화 시스템의 컨텍스트에서 응답을 생성합니다. 입력은 기본 의도/화행 유형과 슬롯(속성) 목록 및 해당 값입니다. 출력은 자연어 문장입니다.

• 다운로드 크기 : 1.46 MiB

• 데이터 세트 크기 : 2.71 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	842
'train'	3,569
'validation'	781

• 기능 구조 :

FeaturesDict({
    'dialog_act': string,
    'dialog_act_delexicalized': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'target_delexicalized': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
dialog_act	텐서		끈
dialog_act_delexicalized	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈
target_delexicalized	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{cs_restaurants,
  address = {Tokyo, Japan},
  title = {Neural {Generation} for {Czech}: {Data} and {Baselines} },
  shorttitle = {Neural {Generation} for {Czech} },
  url = {https://www.aclweb.org/anthology/W19-8670/},
  urldate = {2019-10-18},
  booktitle = {Proceedings of the 12th {International} {Conference} on {Natural} {Language} {Generation} ({INLG} 2019)},
  author = {Dušek, Ondřej and Jurčíček, Filip},
  month = oct,
  year = {2019},
  pages = {563--574}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/다트

• 구성 설명 : DART는 트리 구조 온톨로지를 따르는 엔터티-관계 트리플 세트인 각 입력이 있는 고품질 문장 주석이 있는 대규모 개방형 도메인 구조의 DAta 레코드-텍스트 생성 코퍼스입니다.

• 다운로드 크기 : 28.01 MiB

• 데이터 세트 크기 : 33.78 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	6,959
'train'	62,659
'validation'	2,768

• 기능 구조 :

FeaturesDict({
    'dart_id': int32,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'subtree_was_extended': bool,
    'target': string,
    'target_sources': Sequence(string),
    'tripleset': Sequence(string),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
dart_id	텐서		int32
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
subtree_was_extended	텐서		부울
표적	텐서		끈
target_sources	시퀀스(텐서)	(없음,)	끈
트리플셋	시퀀스(텐서)	(없음,)	끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@article{radev2020dart,
  title=Dart: Open-domain structured data record to text generation,
  author={Radev, Dragomir and Zhang, Rui and Rau, Amrit and Sivaprasad, Abhinand and Hsieh, Chiachun and Rajani, Nazneen Fatema and Tang, Xiangru and Vyas, Aadit and Verma, Neha and Krishna, Pranav and others},
  journal={arXiv preprint arXiv:2007.02871},
  year={2020}
}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/e2e_nlg

• 구성 설명 : E2E 데이터 세트는 최대 8개의 서로 다른 속성(이름, 지역, 가격 범위 등)을 기반으로 레스토랑 설명/추천을 생성하는 제한된 도메인 데이터 대 텍스트 작업을 위해 설계되었습니다.

• 다운로드 크기 : 13.99 MiB

• 데이터 세트 크기 : 16.92 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	4,693
'train'	33,525
'validation'	4,299

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'meaning_representation': string,
    'references': Sequence(string),
    'target': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
의미_표현	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{e2e_cleaned,
  address = {Tokyo, Japan},
  title = {Semantic {Noise} {Matters} for {Neural} {Natural} {Language} {Generation} },
  url = {https://www.aclweb.org/anthology/W19-8652/},
  booktitle = {Proceedings of the 12th {International} {Conference} on {Natural} {Language} {Generation} ({INLG} 2019)},
  author = {Dušek, Ondřej and Howcroft, David M and Rieser, Verena},
  year = {2019},
  pages = {421--426},
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/mlsum_de

• 구성 설명 : MLSum은 대규모 다국어 요약 데이터 세트입니다. 그것은 온라인 뉴스 아울렛에서 만들어졌으며, 이 분할은 독일어에 중점을 둡니다.

• 다운로드 크기 : 345.98 MiB

• 데이터 세트 크기 : 963.60 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'challenge_test_covid'	5,058
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	10,695
'train'	220,748
'validation'	11,392

• 기능 구조 :

FeaturesDict({
    'date': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'text': string,
    'title': string,
    'topic': string,
    'url': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
데이트	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈
텍스트	텐서		끈
제목	텐서		끈
주제	텐서		끈
URL	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{scialom-etal-2020-mlsum,
    title = "{MLSUM}: The Multilingual Summarization Corpus",
    author = {Scialom, Thomas  and Dray, Paul-Alexis  and Lamprier, Sylvain  and Piwowarski, Benjamin  and Staiano, Jacopo},
    booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
    year = {2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/mlsum_es

• 구성 설명 : MLSum은 대규모 다국어 요약 데이터 세트입니다. 온라인 뉴스 매체에서 만들어졌으며 이 분할은 스페인어에 중점을 둡니다.

• 다운로드 크기 : 501.27 MiB

• 데이터세트 크기 : 1.29 GiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'challenge_test_covid'	1,938
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	13,366
'train'	259,888
'validation'	9,977

• 기능 구조 :

FeaturesDict({
    'date': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'text': string,
    'title': string,
    'topic': string,
    'url': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
데이트	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈
텍스트	텐서		끈
제목	텐서		끈
주제	텐서		끈
URL	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{scialom-etal-2020-mlsum,
    title = "{MLSUM}: The Multilingual Summarization Corpus",
    author = {Scialom, Thomas  and Dray, Paul-Alexis  and Lamprier, Sylvain  and Piwowarski, Benjamin  and Staiano, Jacopo},
    booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
    year = {2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/schema_guided_dialog

• 구성 설명 : SGD(스키마 안내 대화) 데이터 세트에는 은행 및 이벤트에서 미디어, 일정, 여행 및 날씨에 이르는 17개 도메인을 다루는 인간과 가상 비서 사이의 18K 다중 도메인 작업 지향 대화가 포함되어 있습니다.

• 다운로드 크기 : 17.00 MiB

• 데이터 세트 크기 : 201.19 MiB

• 자동 캐시 ( 문서 ): 예(challenge_test_backtranslation, challenge_test_bfp02, challenge_test_bfp05, challenge_test_nopunc, challenge_test_scramble, challenge_train_sample, challenge_validation_sample, test, validation), shuffle_files=False (훈련)인 경우에만

• 분할 :

나뉘다	예
'challenge_test_backtranslation'	500
'challenge_test_bfp02'	500
'challenge_test_bfp05'	500
'challenge_test_nopunc'	500
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	10,000
'train'	164,982
'validation'	10,000

• 기능 구조 :

FeaturesDict({
    'context': Sequence(string),
    'dialog_acts': Sequence({
        'act': ClassLabel(shape=(), dtype=int64, num_classes=18),
        'slot': string,
        'values': Sequence(string),
    }),
    'dialog_id': string,
    'gem_id': string,
    'gem_parent_id': string,
    'prompt': string,
    'references': Sequence(string),
    'service': string,
    'target': string,
    'turn_id': int32,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
문맥	시퀀스(텐서)	(없음,)	끈
dialog_acts	순서
dialog_acts/act	클래스 레이블		int64
dialog_acts/슬롯	텐서		끈
dialog_acts/값	시퀀스(텐서)	(없음,)	끈
dialog_id	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
즉각적인	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
서비스	텐서		끈
표적	텐서		끈
turn_id	텐서		int32

• 예 ( tfds.as_dataframe ):

• 인용 :

@article{rastogi2019towards,
  title={Towards Scalable Multi-domain Conversational Agents: The Schema-Guided Dialogue Dataset},
  author={Rastogi, Abhinav and Zang, Xiaoxue and Sunkara, Srinivas and Gupta, Raghav and Khaitan, Pranav},
  journal={arXiv preprint arXiv:1909.05855},
  year={2019}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/토토

• 구성 설명 : ToTTo는 Table-to-Text NLG 작업입니다. 작업은 다음과 같습니다. 행 이름, 열 이름 및 테이블 셀이 있는 Wikipedia 테이블에서 강조 표시된 셀의 하위 집합이 있는 테이블의 강조 표시된 부분에 대한 자연어 설명을 생성합니다.

• 다운로드 크기 : 180.75 MiB

• 데이터 세트 크기 : 645.86 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'challenge_test_scramble'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	7,700
'train'	121,153
'validation'	7,700

• 기능 구조 :

FeaturesDict({
    'example_id': string,
    'gem_id': string,
    'gem_parent_id': string,
    'highlighted_cells': Sequence(Sequence(int32)),
    'overlap_subset': string,
    'references': Sequence(string),
    'sentence_annotations': Sequence({
        'final_sentence': string,
        'original_sentence': string,
        'sentence_after_ambiguity': string,
        'sentence_after_deletion': string,
    }),
    'table': Sequence(Sequence({
        'column_span': int32,
        'is_header': bool,
        'row_span': int32,
        'value': string,
    })),
    'table_page_title': string,
    'table_section_text': string,
    'table_section_title': string,
    'table_webpage_url': string,
    'target': string,
    'totto_id': int32,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
example_id	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
highlight_cells	시퀀스(시퀀스(텐서))	(없음, 없음)	int32
겹침_서브셋	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
문장_주석	순서
sentence_annotations/final_sentence	텐서		끈
sentence_annotations/original_sentence	텐서		끈
sentence_annotations/sentence_after_ambiguity	텐서		끈
sentence_annotations/sentence_after_deletion	텐서		끈
테이블	순서
테이블/column_span	텐서		int32
테이블/is_header	텐서		부울
테이블/row_span	텐서		int32
테이블/값	텐서		끈
table_page_title	텐서		끈
table_section_text	텐서		끈
table_section_title	텐서		끈
table_webpage_url	텐서		끈
표적	텐서		끈
totto_id	텐서		int32

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{parikh2020totto,
  title=ToTTo: A Controlled Table-To-Text Generation Dataset,
  author={Parikh, Ankur and Wang, Xuezhi and Gehrmann, Sebastian and Faruqui, Manaal and Dhingra, Bhuwan and Yang, Diyi and Das, Dipanjan},
  booktitle={Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  pages={1173--1186},
  year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/web_nlg_en

• 구성 설명 : WebNLG는 약 450개의 서로 다른 DBpedia 속성을 다루는 병렬 DBpedia 트리플 세트 및 짧은 텍스트의 이중 언어 데이터 세트(영어, 러시아어)입니다. WebNLG 데이터는 원래 짧은 텍스트를 생성하고 마이크로 계획을 처리할 수 있는 RDF 언어화기의 개발을 촉진하기 위해 만들어졌습니다.

• 다운로드 크기 : 12.57 MiB

• 데이터 세트 크기 : 19.91 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'challenge_test_numbers'	500
'challenge_test_scramble'	500
'challenge_train_sample'	502
'challenge_validation_sample'	499
'test'	1,779
'train'	35,426
'validation'	1,667

• 기능 구조 :

FeaturesDict({
    'category': string,
    'gem_id': string,
    'gem_parent_id': string,
    'input': Sequence(string),
    'references': Sequence(string),
    'target': string,
    'webnlg_id': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
범주	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
입력	시퀀스(텐서)	(없음,)	끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈
webnlg_id	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{gardent2017creating,
  author = "Gardent, Claire
    and Shimorina, Anastasia
    and Narayan, Shashi
    and Perez-Beltrachini, Laura",
  title = "Creating Training Corpora for NLG Micro-Planners",
  booktitle = "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year = "2017",
  publisher = "Association for Computational Linguistics",
  pages = "179--188",
  location = "Vancouver, Canada",
  doi = "10.18653/v1/P17-1017",
  url = "http://www.aclweb.org/anthology/P17-1017"
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/web_nlg_ru

• 구성 설명 : WebNLG는 약 450개의 서로 다른 DBpedia 속성을 다루는 병렬 DBpedia 트리플 세트 및 짧은 텍스트의 이중 언어 데이터 세트(영어, 러시아어)입니다. WebNLG 데이터는 원래 짧은 텍스트를 생성하고 마이크로 계획을 처리할 수 있는 RDF 언어화기의 개발을 촉진하기 위해 만들어졌습니다.

• 다운로드 크기 : 7.49 MiB

• 데이터 세트 크기 : 11.30 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'challenge_test_scramble'	500
'challenge_train_sample'	501
'challenge_validation_sample'	500
'test'	1,102
'train'	14,630
'validation'	790

• 기능 구조 :

FeaturesDict({
    'category': string,
    'gem_id': string,
    'gem_parent_id': string,
    'input': Sequence(string),
    'references': Sequence(string),
    'target': string,
    'webnlg_id': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
범주	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
입력	시퀀스(텐서)	(없음,)	끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈
webnlg_id	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{gardent2017creating,
  author = "Gardent, Claire
    and Shimorina, Anastasia
    and Narayan, Shashi
    and Perez-Beltrachini, Laura",
  title = "Creating Training Corpora for NLG Micro-Planners",
  booktitle = "Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
  year = "2017",
  publisher = "Association for Computational Linguistics",
  pages = "179--188",
  location = "Vancouver, Canada",
  doi = "10.18653/v1/P17-1017",
  url = "http://www.aclweb.org/anthology/P17-1017"
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_auto_asset_turk

• 구성 설명 : WikiAuto는 문장 단순화 시스템을 교육하기 위한 리소스로 영어 Wikipedia 및 Simple English Wikipedia에서 정렬된 문장 세트를 제공합니다. ASSET 및 TURK는 테스트에 사용되는 고품질 단순화 데이터 세트입니다.

• 다운로드 크기 : 121.01 MiB

• 데이터 세트 크기 : 202.40 MiB

• Auto-cached ( documentation ): Yes (challenge_test_asset_backtranslation, challenge_test_asset_bfp02, challenge_test_asset_bfp05, challenge_test_asset_nopunc, challenge_test_turk_backtranslation, challenge_test_turk_bfp02, challenge_test_turk_bfp05, challenge_test_turk_nopunc, challenge_train_sample, challenge_validation_sample, test_asset, test_turk, validation), Only when shuffle_files=False (train)

• 분할 :

나뉘다	예
'challenge_test_asset_backtranslation'	359
'challenge_test_asset_bfp02'	359
'challenge_test_asset_bfp05'	359
'challenge_test_asset_nopunc'	359
'challenge_test_turk_backtranslation'	359
'challenge_test_turk_bfp02'	359
'challenge_test_turk_bfp05'	359
'challenge_test_turk_nopunc'	359
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test_asset'	359
'test_turk'	359
'train'	483,801
'validation'	20,000

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'target': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
표적	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{jiang-etal-2020-neural,
    title = "Neural {CRF} Model for Sentence Alignment in Text Simplification",
    author = "Jiang, Chao  and
      Maddela, Mounica  and
      Lan, Wuwei  and
      Zhong, Yang  and
      Xu, Wei",
    booktitle = "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics",
    month = jul,
    year = "2020",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://www.aclweb.org/anthology/2020.acl-main.709",
    doi = "10.18653/v1/2020.acl-main.709",
    pages = "7943--7960",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/xsum

• 구성 설명 : 데이터 세트는 문서를 한 문장으로 요약하는 극단적인 형태의 추상 요약 작업을 위한 것입니다.

• 다운로드 크기 : 246.31 MiB

• 데이터 세트 크기 : 78.89 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'challenge_test_backtranslation'	500
'challenge_test_bfp_02'	500
'challenge_test_bfp_05'	500
'challenge_test_covid'	401
'challenge_test_nopunc'	500
'challenge_train_sample'	500
'challenge_validation_sample'	500
'test'	1,166
'train'	23,206
'validation'	1,117

• 기능 구조 :

FeaturesDict({
    'document': string,
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'target': string,
    'xsum_id': string,
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
문서	텐서		끈
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
표적	텐서		끈
xsum_id	텐서		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{Narayan2018dont,
  author = "Shashi Narayan and Shay B. Cohen and Mirella Lapata",
  title = "Don't Give Me the Details, Just the Summary! {T}opic-Aware Convolutional Neural Networks for Extreme Summarization",
  booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing ",
  year = "2018",
  address = "Brussels, Belgium",
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_arabic_ar

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 56.25 MiB

• 데이터 세트 크기 : 291.42 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	5,841
'train'	20,441
'validation'	2,919

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'ar': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'ar': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/ar	텍스트		끈
source_aligned/en	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/ar	텍스트		끈
target_aligned/en	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_chinese_zh

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 31.38 MiB

• 데이터 세트 크기 : 122.06 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	3,775
'train'	13,211
'validation'	1,886

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'zh': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'zh': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/zh	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/zh	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_czech_cs

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 13.84 MiB

• 데이터 세트 크기 : 58.05 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	1,438
'train'	5,033
'validation'	718

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'cs': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'cs': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/cs	텍스트		끈
source_aligned/en	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/cs	텍스트		끈
target_aligned/en	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_dutch_nl

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 53.88 MiB

• 데이터 세트 크기 : 237.97 MiB

• 자동 캐싱 ( 문서 ): 예(테스트, 검증), shuffle_files=False 인 경우에만(훈련)

• 분할 :

나뉘다	예
'test'	6,248
'train'	21,866
'validation'	3,123

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'nl': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'nl': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/nl	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/nl	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_english_en

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 112.56 MiB

• 데이터 세트 크기 : 657.51 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	28,614
'train'	99,020
'validation'	13,823

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_french_fr

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 113.26 MiB

• 데이터 세트 크기 : 522.28 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	12,731
'train'	44,556
'validation'	6,364

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'fr': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'fr': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/fr	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/fr	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_german_de

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 102.65 MiB

• 데이터 세트 크기 : 452.46 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	11,669
'train'	40,839
'validation'	5,833

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'de': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'de': Text(shape=(), dtype=string),
        'en': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/de	텍스트		끈
source_aligned/en	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/de	텍스트		끈
target_aligned/en	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_hindi_hi

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 20.07 MiB

• 데이터 세트 크기 : 138.06 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	1,984
'train'	6,942
'validation'	991

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'hi': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'hi': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/hi	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/hi	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_indonesian_id

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 80.08 MiB

• 데이터 세트 크기 : 370.63 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	9,497
'train'	33,237
'validation'	4,747

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'id': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'id': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/id	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/id	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_italian_it

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 84.80 MiB

• 데이터 세트 크기 : 374.40 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	10,189
'train'	35,661
'validation'	5,093

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'it': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'it': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/it	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/it	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_japanese_ja

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 21.75 MiB

• 데이터 세트 크기 : 103.19 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	2,530
'train'	8,853
'validation'	1,264

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ja': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ja': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/ja	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/ja	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

gem/wiki_lingua_korean_ko

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 22.26 MiB

• 데이터 세트 크기 : 102.35 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	2,436
'train'	8,524
'validation'	1,216

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ko': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ko': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/ko	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/코	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_portuguese_pt

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 131.17 MiB

• 데이터 세트 크기 : 570.46 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	16,331
'train'	57,159
'validation'	8,165

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'pt': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'pt': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
소스_정렬/pt	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/pt	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_russian_ru

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 101.36 MiB

• 데이터 세트 크기 : 564.69 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	10,580
'train'	37,028
'validation'	5,288

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ru': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'ru': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/ru	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/ru	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_spanish_es

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 189.06 MiB

• 데이터 세트 크기 : 849.75 MiB

• 자동 캐시 ( 문서 ): 아니요

• 분할 :

나뉘다	예
'test'	22,632
'train'	79,212
'validation'	11,316

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'es': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'es': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/es	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/es	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_thai_th

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 28.60 MiB

• 데이터 세트 크기 : 193.77 MiB

• 자동 캐싱 ( 문서 ): 예(테스트, 검증), shuffle_files=False 인 경우에만(훈련)

• 분할 :

나뉘다	예
'test'	2,950
'train'	10,325
'validation'	1,475

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'th': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'th': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/th	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/th	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_turkish_tr

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 6.73 MiB

• 데이터 세트 크기 : 30.75 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	900
'train'	3,148
'validation'	449

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'tr': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'tr': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/tr	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/tr	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."

보석/wiki_lingua_vietnamese_vi

• 구성 설명 : Wikilingua는 언어 간 추상 요약 시스템 평가를 위한 대규모 다국어 데이터 세트입니다.

• 다운로드 크기 : 36.27 MiB

• 데이터 세트 크기 : 179.77 MiB

• 자동 캐시 ( 문서 ): 예

• 분할 :

나뉘다	예
'test'	3,917
'train'	13,707
'validation'	1,957

• 기능 구조 :

FeaturesDict({
    'gem_id': string,
    'gem_parent_id': string,
    'references': Sequence(string),
    'source': string,
    'source_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'vi': Text(shape=(), dtype=string),
    }),
    'target': string,
    'target_aligned': Translation({
        'en': Text(shape=(), dtype=string),
        'vi': Text(shape=(), dtype=string),
    }),
})

• 기능 문서 :

특징	수업	모양	D타입	설명
	풍모Dict
gem_id	텐서		끈
gem_parent_id	텐서		끈
참조	시퀀스(텐서)	(없음,)	끈
원천	텐서		끈
source_aligned	번역
source_aligned/en	텍스트		끈
source_aligned/vi	텍스트		끈
표적	텐서		끈
target_aligned	번역
target_aligned/en	텍스트		끈
target_aligned/vi	텍스트		끈

• 예 ( tfds.as_dataframe ):

• 인용 :

@inproceedings{ladhak-wiki-2020,
title=WikiLingua: A New Benchmark Dataset for Multilingual Abstractive Summarization,
author={Faisal Ladhak, Esin Durmus, Claire Cardie and Kathleen McKeown},
booktitle={Findings of EMNLP, 2020},
year={2020}
}
@article{gehrmann2021gem,
  author    = {Sebastian Gehrmann and
               Tosin P. Adewumi and
               Karmanya Aggarwal and
               Pawan Sasanka Ammanamanchi and
               Aremu Anuoluwapo and
               Antoine Bosselut and
               Khyathi Raghavi Chandu and
               Miruna{-}Adriana Clinciu and
               Dipanjan Das and
               Kaustubh D. Dhole and
               Wanyu Du and
               Esin Durmus and
               Ondrej Dusek and
               Chris Emezue and
               Varun Gangal and
               Cristina Garbacea and
               Tatsunori Hashimoto and
               Yufang Hou and
               Yacine Jernite and
               Harsh Jhamtani and
               Yangfeng Ji and
               Shailza Jolly and
               Dhruv Kumar and
               Faisal Ladhak and
               Aman Madaan and
               Mounica Maddela and
               Khyati Mahajan and
               Saad Mahamood and
               Bodhisattwa Prasad Majumder and
               Pedro Henrique Martins and
               Angelina McMillan{-}Major and
               Simon Mille and
               Emiel van Miltenburg and
               Moin Nadeem and
               Shashi Narayan and
               Vitaly Nikolaev and
               Rubungo Andre Niyongabo and
               Salomey Osei and
               Ankur P. Parikh and
               Laura Perez{-}Beltrachini and
               Niranjan Ramesh Rao and
               Vikas Raunak and
               Juan Diego Rodriguez and
               Sashank Santhanam and
               Jo{\~{a} }o Sedoc and
               Thibault Sellam and
               Samira Shaikh and
               Anastasia Shimorina and
               Marco Antonio Sobrevilla Cabezudo and
               Hendrik Strobelt and
               Nishant Subramani and
               Wei Xu and
               Diyi Yang and
               Akhila Yerukola and
               Jiawei Zhou},
  title     = {The {GEM} Benchmark: Natural Language Generation, its Evaluation and
               Metrics},
  journal   = {CoRR},
  volume    = {abs/2102.01672},
  year      = {2021},
  url       = {https://arxiv.org/abs/2102.01672},
  archivePrefix = {arXiv},
  eprint    = {2102.01672}
}

Note that each GEM dataset has its own citation. Please see the source to see
the correct citation for each contained dataset."