参考文献:
バックエル
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:emea/bg-el')
- 説明：
This is a parallel corpus made out of PDF documents from the European Medicines Agency. All files are automatically converted from PDF to plain text using pdftotext with the command line arguments -layout -nopgbrk -eol unix. There are some known problems with tables and multi-column layouts - some of them are fixed in the current version.
source: http://www.emea.europa.eu/
22 languages, 231 bitexts
total number of files: 41,957
total number of tokens: 311.65M
total number of sentence fragments: 26.51M
- ライセンス: 不明なライセンス
- バージョン: 3.0.0
- 分割:
| スプリット | 例 |
|---|---|
'train' | 1044065 |
- 特徴：
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"translation": {
"languages": [
"bg",
"el"
],
"id": null,
"_type": "Translation"
}
}
cs-et
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:emea/cs-et')
- 説明：
This is a parallel corpus made out of PDF documents from the European Medicines Agency. All files are automatically converted from PDF to plain text using pdftotext with the command line arguments -layout -nopgbrk -eol unix. There are some known problems with tables and multi-column layouts - some of them are fixed in the current version.
source: http://www.emea.europa.eu/
22 languages, 231 bitexts
total number of files: 41,957
total number of tokens: 311.65M
total number of sentence fragments: 26.51M
- ライセンス: 不明なライセンス
- バージョン: 3.0.0
- 分割:
| スプリット | 例 |
|---|---|
'train' | 1053164 |
- 特徴：
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"translation": {
"languages": [
"cs",
"et"
],
"id": null,
"_type": "Translation"
}
}
デマウント
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:emea/de-mt')
- 説明：
This is a parallel corpus made out of PDF documents from the European Medicines Agency. All files are automatically converted from PDF to plain text using pdftotext with the command line arguments -layout -nopgbrk -eol unix. There are some known problems with tables and multi-column layouts - some of them are fixed in the current version.
source: http://www.emea.europa.eu/
22 languages, 231 bitexts
total number of files: 41,957
total number of tokens: 311.65M
total number of sentence fragments: 26.51M
- ライセンス: 不明なライセンス
- バージョン: 3.0.0
- 分割:
| スプリット | 例 |
|---|---|
'train' | 1000532 |
- 特徴：
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"translation": {
"languages": [
"de",
"mt"
],
"id": null,
"_type": "Translation"
}
}
フロスク
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:emea/fr-sk')
- 説明：
This is a parallel corpus made out of PDF documents from the European Medicines Agency. All files are automatically converted from PDF to plain text using pdftotext with the command line arguments -layout -nopgbrk -eol unix. There are some known problems with tables and multi-column layouts - some of them are fixed in the current version.
source: http://www.emea.europa.eu/
22 languages, 231 bitexts
total number of files: 41,957
total number of tokens: 311.65M
total number of sentence fragments: 26.51M
- ライセンス: 不明なライセンス
- バージョン: 3.0.0
- 分割:
| スプリット | 例 |
|---|---|
'train' | 1062753 |
- 特徴：
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"translation": {
"languages": [
"fr",
"sk"
],
"id": null,
"_type": "Translation"
}
}
es-lt
次のコマンドを使用して、このデータセットを TFDS にロードします。
ds = tfds.load('huggingface:emea/es-lt')
- 説明：
This is a parallel corpus made out of PDF documents from the European Medicines Agency. All files are automatically converted from PDF to plain text using pdftotext with the command line arguments -layout -nopgbrk -eol unix. There are some known problems with tables and multi-column layouts - some of them are fixed in the current version.
source: http://www.emea.europa.eu/
22 languages, 231 bitexts
total number of files: 41,957
total number of tokens: 311.65M
total number of sentence fragments: 26.51M
- ライセンス: 不明なライセンス
- バージョン: 3.0.0
- 分割:
| スプリット | 例 |
|---|---|
'train' | 1051370 |
- 特徴：
{
"id": {
"dtype": "string",
"id": null,
"_type": "Value"
},
"translation": {
"languages": [
"es",
"lt"
],
"id": null,
"_type": "Translation"
}
}