大規模なスペイン語コーパス

参考文献:

JRC

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/JRC')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 3410620
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

EMEA

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/EMEA')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 1221233
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

グローバルボイス

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/GlobalVoices')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 897075
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

ECB

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/ECB')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 1875738
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

犬

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/DOGC')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 10917053
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

すべてのウィキ

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/all_wikis')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 28109484
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

テッド

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/TED')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 157910
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

マルチ国連

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/multiUN')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 13127490
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

ユーロパール

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/Europarl')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 2174141
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

ニュース解説11

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/NewsCommentary11')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 288771
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

国連

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/UN')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 74067
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

EUブックショップ

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/EUBookShop')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 8214959
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

パラクロール

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/ParaCrawl')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 15510649
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

オープン字幕2018

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/OpenSubtitles2018')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 213508602
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

DGT

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/DGT')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 3168368
  • 特徴： 
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}

組み合わせた

次のコマンドを使用して、このデータセットを TFDS にロードします。

ds = tfds.load('huggingface:large_spanish_corpus/combined')
  • 説明：
The Large Spanish Corpus is a compilation of 15 unlabelled Spanish corpora spanning Wikipedia to European parliament notes. Each config contains the data corresponding to a different corpus. For example, "all_wiki" only includes examples from Spanish Wikipedia. By default, the config is set to "combined" which loads all the corpora; with this setting you can also specify the number of samples to return per corpus by configuring the "split" argument.
  • ライセンス: MIT
  • バージョン: 1.1.0
  • 分割:
スプリット例
'train' 302656160
  • 特徴：
{
    "text": {
        "dtype": "string",
        "id": null,
        "_type": "Value"
    }
}