参考文献:
wnli.jp
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wnli.en')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Winograd Schema Challenge (Levesque et al., 2011) is a reading comprehension task
in which a system must read a sentence with a pronoun and select the referent of that pronoun from
a list of choices. The examples are manually constructed to foil simple statistical methods: Each
one is contingent on contextual information provided by a single word or phrase in the sentence.
To convert the problem into sentence pair classification, we construct sentence pairs by replacing
the ambiguous pronoun with each possible referent. The task is to predict if the sentence with the
pronoun substituted is entailed by the original sentence. We use a small evaluation set consisting of
new examples derived from fiction books that was shared privately by the authors of the original
corpus. While the included training set is balanced between two classes, the test set is imbalanced
between them (65% not entailment). Also, due to a data quirk, the development set is adversarial:
hypotheses are sometimes shared between training and development examples, so if a model memorizes the
training examples, they will predict the wrong label on corresponding development set
example. As with QNLI, each example is evaluated separately, so there is not a systematic correspondence
between a model's score on this task and its score on the unconverted original task. We
call converted dataset WNLI (Winograd NLI). This dataset is translated and publicly released for 3
Indian languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 146 |
'train' | 635 |
'validation' | 71 |
- 特徴：
{
"hypothesis": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 3,
"names": [
"not_entailment",
"entailment",
"None"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
wnli.こんにちは
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wnli.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Winograd Schema Challenge (Levesque et al., 2011) is a reading comprehension task
in which a system must read a sentence with a pronoun and select the referent of that pronoun from
a list of choices. The examples are manually constructed to foil simple statistical methods: Each
one is contingent on contextual information provided by a single word or phrase in the sentence.
To convert the problem into sentence pair classification, we construct sentence pairs by replacing
the ambiguous pronoun with each possible referent. The task is to predict if the sentence with the
pronoun substituted is entailed by the original sentence. We use a small evaluation set consisting of
new examples derived from fiction books that was shared privately by the authors of the original
corpus. While the included training set is balanced between two classes, the test set is imbalanced
between them (65% not entailment). Also, due to a data quirk, the development set is adversarial:
hypotheses are sometimes shared between training and development examples, so if a model memorizes the
training examples, they will predict the wrong label on corresponding development set
example. As with QNLI, each example is evaluated separately, so there is not a systematic correspondence
between a model's score on this task and its score on the unconverted original task. We
call converted dataset WNLI (Winograd NLI). This dataset is translated and publicly released for 3
Indian languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 146 |
'train' | 635 |
'validation' | 71 |
- 特徴：
{
"hypothesis": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 3,
"names": [
"not_entailment",
"entailment",
"None"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
wnli.gu
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wnli.gu')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Winograd Schema Challenge (Levesque et al., 2011) is a reading comprehension task
in which a system must read a sentence with a pronoun and select the referent of that pronoun from
a list of choices. The examples are manually constructed to foil simple statistical methods: Each
one is contingent on contextual information provided by a single word or phrase in the sentence.
To convert the problem into sentence pair classification, we construct sentence pairs by replacing
the ambiguous pronoun with each possible referent. The task is to predict if the sentence with the
pronoun substituted is entailed by the original sentence. We use a small evaluation set consisting of
new examples derived from fiction books that was shared privately by the authors of the original
corpus. While the included training set is balanced between two classes, the test set is imbalanced
between them (65% not entailment). Also, due to a data quirk, the development set is adversarial:
hypotheses are sometimes shared between training and development examples, so if a model memorizes the
training examples, they will predict the wrong label on corresponding development set
example. As with QNLI, each example is evaluated separately, so there is not a systematic correspondence
between a model's score on this task and its score on the unconverted original task. We
call converted dataset WNLI (Winograd NLI). This dataset is translated and publicly released for 3
Indian languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 146 |
'train' | 635 |
'validation' | 71 |
- 特徴：
{
"hypothesis": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 3,
"names": [
"not_entailment",
"entailment",
"None"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
wnli.mr
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wnli.mr')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Winograd Schema Challenge (Levesque et al., 2011) is a reading comprehension task
in which a system must read a sentence with a pronoun and select the referent of that pronoun from
a list of choices. The examples are manually constructed to foil simple statistical methods: Each
one is contingent on contextual information provided by a single word or phrase in the sentence.
To convert the problem into sentence pair classification, we construct sentence pairs by replacing
the ambiguous pronoun with each possible referent. The task is to predict if the sentence with the
pronoun substituted is entailed by the original sentence. We use a small evaluation set consisting of
new examples derived from fiction books that was shared privately by the authors of the original
corpus. While the included training set is balanced between two classes, the test set is imbalanced
between them (65% not entailment). Also, due to a data quirk, the development set is adversarial:
hypotheses are sometimes shared between training and development examples, so if a model memorizes the
training examples, they will predict the wrong label on corresponding development set
example. As with QNLI, each example is evaluated separately, so there is not a systematic correspondence
between a model's score on this task and its score on the unconverted original task. We
call converted dataset WNLI (Winograd NLI). This dataset is translated and publicly released for 3
Indian languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 146 |
'train' | 635 |
'validation' | 71 |
- 特徴：
{
"hypothesis": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 3,
"names": [
"not_entailment",
"entailment",
"None"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
コパエン
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/copa.en')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Choice Of Plausible Alternatives (COPA) evaluation provides researchers with a tool for assessing
progress in open-domain commonsense causal reasoning. COPA consists of 1000 questions, split equally
into development and test sets of 500 questions each. Each question is composed of a premise and two
alternatives, where the task is to select the alternative that more plausibly has a causal relation
with the premise. The correct alternative is randomized so that the expected performance of randomly
guessing is 50%. This dataset is translated and publicly released for 3 languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 500 |
'train' | 400 |
'validation' | 100 |
- 特徴：
{
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice2": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
コパ・ハイ
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/copa.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Choice Of Plausible Alternatives (COPA) evaluation provides researchers with a tool for assessing
progress in open-domain commonsense causal reasoning. COPA consists of 1000 questions, split equally
into development and test sets of 500 questions each. Each question is composed of a premise and two
alternatives, where the task is to select the alternative that more plausibly has a causal relation
with the premise. The correct alternative is randomized so that the expected performance of randomly
guessing is 50%. This dataset is translated and publicly released for 3 languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 449 |
'train' | 362 |
'validation' | 88 |
- 特徴：
{
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice2": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
コパグ
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/copa.gu')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Choice Of Plausible Alternatives (COPA) evaluation provides researchers with a tool for assessing
progress in open-domain commonsense causal reasoning. COPA consists of 1000 questions, split equally
into development and test sets of 500 questions each. Each question is composed of a premise and two
alternatives, where the task is to select the alternative that more plausibly has a causal relation
with the premise. The correct alternative is randomized so that the expected performance of randomly
guessing is 50%. This dataset is translated and publicly released for 3 languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 448 |
'train' | 362 |
'validation' | 88 |
- 特徴：
{
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice2": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
コパミスター
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/copa.mr')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Choice Of Plausible Alternatives (COPA) evaluation provides researchers with a tool for assessing
progress in open-domain commonsense causal reasoning. COPA consists of 1000 questions, split equally
into development and test sets of 500 questions each. Each question is composed of a premise and two
alternatives, where the task is to select the alternative that more plausibly has a causal relation
with the premise. The correct alternative is randomized so that the expected performance of randomly
guessing is 50%. This dataset is translated and publicly released for 3 languages by AI4Bharat.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 449 |
'train' | 362 |
'validation' | 88 |
- 特徴：
{
"premise": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"choice2": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
sna.bn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/sna.bn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
This dataset is a collection of Bengali News articles. The dataset is used for classifying articles into
5 different classes namely international, state, kolkata, entertainment and sports.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1411 |
'train' | 11284 |
'validation' | 1411 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 6,
"names": [
"kolkata",
"state",
"national",
"sports",
"entertainment",
"international"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
csqa.as
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.as')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 2942 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.bn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.bn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 38845 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.gu
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.gu')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 22861 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.こんにちは
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 35140 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.kn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.kn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 13666 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.ml
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.ml')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 26537 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.mr
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.mr')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 11370 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.or
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.or')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1975年 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.pa
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.pa')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5667 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.ta
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.ta')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 38590 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
csqa.te
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/csqa.te')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Given a text with an entity randomly masked, the task is to predict that masked entity from a list of 4
candidate entities. The dataset contains around 239k examples across 11 languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 41338 |
- 特徴：
{
"question": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"answer": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"category": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"title": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"out_of_context_options": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
wstp.as
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.as')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 626 |
'train' | 5000 |
'validation' | 625 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.bn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.bn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5948 |
'train' | 47580 |
'validation' | 5947 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.gu
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.gu')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1251 |
'train' | 10004 |
'validation' | 1251 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.hi
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5509 |
'train' | 44069 |
'validation' | 5509 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.kn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.kn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 4423 |
'train' | 35379 |
'validation' | 4422 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.ml
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.ml')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 3441 |
'train' | 27527 |
'validation' | 3441 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.mr
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.mr')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1306 |
'train' | 10446 |
'validation' | 1306 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.or
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.or')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 502 |
'train' | 4015 |
'validation' | 502 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.pa
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.pa')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1097 |
'train' | 8772 |
'validation' | 1097 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.ta
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.ta')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 6118 |
'train' | 48940 |
'validation' | 6117 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
wstp.te
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wstp.te')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Predict the correct title for a Wikipedia section from a given list of four candidate titles.
The dataset has 400k examples across 11 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 10000 |
'train' | 80000 |
'validation' | 10000 |
- 特徴：
{
"sectionText": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"correctTitle": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleA": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleB": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleC": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"titleD": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"url": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
inltkh.gu
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/inltkh.gu')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Obtained from inltk project. The corpus is a collection of headlines tagged with their news category.
Available for langauges: gu, ml, mr and ta.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 659 |
'train' | 5269 |
'validation' | 659 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 10,
"names": [
"entertainment",
"business",
"tech",
"sports",
"state",
"spirituality",
"tamil-cinema",
"positive",
"negative",
"neutral"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
inltkh.ml
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/inltkh.ml')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Obtained from inltk project. The corpus is a collection of headlines tagged with their news category.
Available for langauges: gu, ml, mr and ta.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 630 |
'train' | 5036 |
'validation' | 630 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 10,
"names": [
"entertainment",
"business",
"tech",
"sports",
"state",
"spirituality",
"tamil-cinema",
"positive",
"negative",
"neutral"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
inltkh.mr
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/inltkh.mr')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Obtained from inltk project. The corpus is a collection of headlines tagged with their news category.
Available for langauges: gu, ml, mr and ta.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1210 |
'train' | 9672 |
'validation' | 1210 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 10,
"names": [
"entertainment",
"business",
"tech",
"sports",
"state",
"spirituality",
"tamil-cinema",
"positive",
"negative",
"neutral"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
inltkh.ta
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/inltkh.ta')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Obtained from inltk project. The corpus is a collection of headlines tagged with their news category.
Available for langauges: gu, ml, mr and ta.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 669 |
'train' | 5346 |
'validation' | 669 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 10,
"names": [
"entertainment",
"business",
"tech",
"sports",
"state",
"spirituality",
"tamil-cinema",
"positive",
"negative",
"neutral"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
inltkh.te
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/inltkh.te')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
Obtained from inltk project. The corpus is a collection of headlines tagged with their news category.
Available for langauges: gu, ml, mr and ta.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 541 |
'train' | 4328 |
'validation' | 541 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 10,
"names": [
"entertainment",
"business",
"tech",
"sports",
"state",
"spirituality",
"tamil-cinema",
"positive",
"negative",
"neutral"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
bbca.こんにちは
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/bbca.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
This release consists of 4335 Hindi documents with tags from the BBC Hindi News website.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 866 |
'train' | 3467 |
- 特徴：
{
"label": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-bn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-bn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5522 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-gu
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-gu')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 6463 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-hi
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5169 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-ml
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-ml')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 4886 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-mr
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-mr')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5760 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-or
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-or')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 752 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-ta
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-ta')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5637 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-te
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-te')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 5049 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
cvit-mkb-clsr.en-ur
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/cvit-mkb-clsr.en-ur')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
CVIT Maan ki Baat Dataset - Given a sentence in language $L_1$ the task is to retrieve its translation
from a set of candidate sentences in language $L_2$.
The dataset contains around 39k parallel sentence pairs across 8 Indian languages.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1006 |
- 特徴：
{
"sentence1": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence2": {
"dtype": "string",
"id": null,
"_type": "Value"
}
}
iitpさんこんにちは
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/iitp-mr.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
IIT Patna Product Reviews: Sentiment analysis corpus for product reviews posted in Hindi.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 310 |
'train' | 2480 |
'validation' | 310 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 3,
"names": [
"negative",
"neutral",
"positive"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
iitp-pr.hi
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/iitp-pr.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
IIT Patna Product Reviews: Sentiment analysis corpus for product reviews posted in Hindi.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 523 |
'train' | 4182 |
'validation' | 523 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 3,
"names": [
"negative",
"neutral",
"positive"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
アクタSC.TE
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/actsa-sc.te')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
ACTSA Corpus: Sentiment analysis corpus for Telugu sentences.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 541 |
'train' | 4328 |
'validation' | 541 |
- 特徴：
{
"text": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"label": {
"num_classes": 2,
"names": [
"positive",
"negative"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
}
}
MD.こんにちは
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/md.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The Hindi Discourse Analysis dataset is a corpus for analyzing discourse modes present in its sentences.
It contains sentences from stories written by 11 famous authors from the 20th Century. 4-5 stories by
each author have been selected which were available in the public domain resulting in a collection of 53 stories.
Most of these short stories were originally written in Hindi but some of them were written in other Indian languages
and later translated to Hindi.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 997 |
'train' | 7974 |
'validation' | 997 |
- 特徴：
{
"sentence": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"discourse_mode": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"story_number": {
"dtype": "int32",
"id": null,
"_type": "Value"
},
"id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
ウィキナー.as
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.as')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 160 |
'train' | 1021 |
'validation' | 157 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
ウィキナー.bn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.bn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 2690 |
'train' | 20223 |
'validation' | 2985 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
wiki-ner.gu
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.gu')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 255 |
'train' | 2343 |
'validation' | 297 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
wikiner.hi
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.hi')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1256 |
'train' | 9463 |
'validation' | 1114 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
ウィキナー.kn
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.kn')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 476 |
'train' | 2679 |
'validation' | 412 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
ウィキナー.ml
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.ml')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 2042年 |
'train' | 15620 |
'validation' | 2067年 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
ウィキナー.mr
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.mr')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1329 |
'train' | 12151 |
'validation' | 1498年 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
wikiner.or
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.or')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 153 |
'train' | 1077 |
'validation' | 132 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
wiki-ner.pa
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.pa')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 179 |
'train' | 1408 |
'validation' | 186 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
wiki-ner.ta
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.ta')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 2611 |
'train' | 20466 |
'validation' | 2586 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}
ウィキナー.te
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:indic_glue/wiki-ner.te')
- 説明：
IndicGLUE is a natural language understanding benchmark for Indian languages. It contains a wide
variety of tasks and covers 11 major Indian languages - as, bn, gu, hi, kn, ml, mr, or, pa, ta, te.
The WikiANN dataset (Pan et al. 2017) is a dataset with NER annotations for PER, ORG and LOC. It has been constructed using
the linked entities in Wikipedia pages for 282 different languages including Danish.
- ライセンス: 既知のライセンスはありません
- バージョン: 1.0.0
- 分割:
| スプリット | 例 |
|---|---|
'test' | 1110 |
'train' | 7978 |
'validation' | 841 |
- 特徴：
{
"tokens": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"ner_tags": {
"feature": {
"num_classes": 7,
"names": [
"B-LOC",
"B-ORG",
"B-PER",
"I-LOC",
"I-ORG",
"I-PER",
"O"
],
"names_file": null,
"id": null,
"_type": "ClassLabel"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"additional_info": {
"feature": {
"feature": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"length": -1,
"id": null,
"_type": "Sequence"
},
"length": -1,
"id": null,
"_type": "Sequence"
}
}