References:
adjunct_island
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/adjunct_island')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
anaphor_gender_agreement
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/anaphor_gender_agreement')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
anaphor_number_agreement
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/anaphor_number_agreement')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
animate_subject_passive
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/animate_subject_passive')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
animate_subject_trans
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/animate_subject_trans')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
causative
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/causative')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
complex_NP_island
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/complex_NP_island')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
coordinate_structure_constraint_complex_left_branch
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/coordinate_structure_constraint_complex_left_branch')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
coordinate_structure_constraint_object_extraction
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/coordinate_structure_constraint_object_extraction')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_1
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_1')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_2
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_2')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_irregular_1
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_irregular_1')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_irregular_2
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_irregular_2')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_with_adj_2
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_with_adj_2')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_with_adj_irregular_1
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_with_adj_irregular_1')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_with_adj_irregular_2
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_with_adj_irregular_2')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
determiner_noun_agreement_with_adjective_1
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/determiner_noun_agreement_with_adjective_1')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
distractor_agreement_relational_noun
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/distractor_agreement_relational_noun')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
distractor_agreement_relative_clause
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/distractor_agreement_relative_clause')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
drop_argument
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/drop_argument')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
ellipsis_n_bar_1
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/ellipsis_n_bar_1')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
ellipsis_n_bar_2
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/ellipsis_n_bar_2')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
existential_there_object_raising
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/existential_there_object_raising')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
existential_there_quantifiers_1
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/existential_there_quantifiers_1')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
existential_there_quantifiers_2
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/existential_there_quantifiers_2')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
existential_there_subject_raising
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/existential_there_subject_raising')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
expletive_it_object_raising
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/expletive_it_object_raising')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
inchoative
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/inchoative')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
intransitive
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/intransitive')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
irregular_past_participle_adjectives
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/irregular_past_participle_adjectives')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
irregular_past_participle_verbs
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/irregular_past_participle_verbs')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
irregular_plural_subject_verb_agreement_1
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/irregular_plural_subject_verb_agreement_1')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
irregular_plural_subject_verb_agreement_2
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/irregular_plural_subject_verb_agreement_2')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
left_branch_island_echo_question
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/left_branch_island_echo_question')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}
}
left_branch_island_simple_question
Use the following command to load this dataset in TFDS:
ds = tfds.load('huggingface:blimp/left_branch_island_simple_question')
- Description:
BLiMP is a challenge set for evaluating what language models (LMs) know about
major grammatical phenomena in English. BLiMP consists of 67 sub-datasets, each
containing 1000 minimal pairs isolating specific contrasts in syntax,
morphology, or semantics. The data is automatically generated according to
expert-crafted grammars.
- License: No known license
- Version: 0.1.0
- Splits:
| Split | Examples |
|---|---|
'train' |
1000 |
- Features:
{
"sentence_good": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"sentence_bad": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"field": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"linguistics_term": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"UID": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"simple_LM_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"one_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"two_prefix_method": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"lexically_identical": {
"dtype": "bool",
"id": null,
"_type": "Value"
},
"pair_id": {
"dtype": "int32",
"id": null,
"_type": "Value"
}