tfds.folder_dataset.write_metadata
Stay organized with collections
Save and categorize content based on your preferences.
Add metadata required to load with TFDS.
tfds.folder_dataset.write_metadata(
*,
data_dir: tfds.typing.PathLike,
features: features_lib.feature.FeatureConnectorArg,
split_infos: Union[None, epath.PathLike, List[split_lib.SplitInfo]] = None,
version: Union[None, str, tfds.core.Version] = None,
filename_template: Union[None, str, tfds.core.ShardedFileTemplate] = None,
check_data: bool = True,
**ds_info_kwargs
) -> None
See documentation for usage:
https://www.tensorflow.org/datasets/external_tfrecord
Args |
|---|
data_dir
|
Dataset path on which to save the metadata
|
features
|
dict of tfds.features.FeatureConnector matching the proto specs.
|
split_infos
|
Can be either: * A path to the pre-computed split info values
( the out_dir kwarg of tfds.folder_dataset.compute_split_info) * A
list of tfds.core.SplitInfo (returned value of
tfds.folder_dataset.compute_split_info) * None to auto-compute the
split info.
|
version
|
Optional dataset version (auto-infer by default, or fallback to
1.0.0)
|
filename_template
|
the template for the filenames of the data. If None, then
the default template '{DATASET}-{SPLIT}.{FILEFORMAT}-{SHARD_X_OF_Y}' is
used. A string or a ShardedFileTemplate can be given for custom templates.
|
check_data
|
If True, perform additional check to validate the data in
data_dir is valid
|
**ds_info_kwargs
|
Additional metadata forwarded to tfds.core.DatasetInfo (
description, homepage,...). Will appear in the doc.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-04-26 UTC.
[null,null,["Last updated 2024-04-26 UTC."],[],[],null,["# tfds.folder_dataset.write_metadata\n\n\u003cbr /\u003e\n\n|-------------------------------------------------------------------------------------------------------------------------------------------------------|\n| [View source on GitHub](https://github.com/tensorflow/datasets/blob/v4.9.3/tensorflow_datasets/core/folder_dataset/write_metadata_utils.py#L105-L191) |\n\nAdd metadata required to load with TFDS. \n\n tfds.folder_dataset.write_metadata(\n *,\n data_dir: ../../tfds/typing/PathLike,\n features: features_lib.feature.FeatureConnectorArg,\n split_infos: Union[None, epath.PathLike, List[split_lib.SplitInfo]] = None,\n version: Union[None, str, ../../tfds/core/Version] = None,\n filename_template: Union[None, str, ../../tfds/core/ShardedFileTemplate] = None,\n check_data: bool = True,\n **ds_info_kwargs\n ) -\u003e None\n\nSee documentation for usage:\n\u003chttps://www.tensorflow.org/datasets/external_tfrecord\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|---------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|\n| `data_dir` | Dataset path on which to save the metadata |\n| `features` | dict of [`tfds.features.FeatureConnector`](../../tfds/features/FeatureConnector) matching the proto specs. |\n| `split_infos` | Can be either: \\* A path to the pre-computed split info values ( the `out_dir` kwarg of [`tfds.folder_dataset.compute_split_info`](../../tfds/folder_dataset/compute_split_info)) \\* A list of [`tfds.core.SplitInfo`](../../tfds/core/SplitInfo) (returned value of [`tfds.folder_dataset.compute_split_info`](../../tfds/folder_dataset/compute_split_info)) \\* `None` to auto-compute the split info. |\n| `version` | Optional dataset version (auto-infer by default, or fallback to 1.0.0) |\n| `filename_template` | the template for the filenames of the data. If None, then the default template `'{DATASET}-{SPLIT}.{FILEFORMAT}-{SHARD_X_OF_Y}'` is used. A string or a ShardedFileTemplate can be given for custom templates. |\n| `check_data` | If True, perform additional check to validate the data in data_dir is valid |\n| `**ds_info_kwargs` | Additional metadata forwarded to [`tfds.core.DatasetInfo`](../../tfds/core/DatasetInfo) ( description, homepage,...). Will appear in the doc. |\n\n\u003cbr /\u003e"]]
View source on GitHub