tf.raw_ops.SnapshotDatasetV2
Stay organized with collections
Save and categorize content based on your preferences.
Creates a dataset that will write to / read from a snapshot.
tf.raw_ops.SnapshotDatasetV2(
input_dataset,
path,
reader_func_other_args,
shard_func_other_args,
output_types,
output_shapes,
reader_func,
shard_func,
compression='',
reader_prefix='',
writer_prefix='',
hash_valid=False,
hash=0,
metadata='',
name=None
)
This dataset attempts to determine whether a valid snapshot exists at the
snapshot_path, and reads from the snapshot in lieu of using input_dataset.
If not, it will run the preprocessing pipeline as usual, and write out a
snapshot of the data processed for future use.
Args |
|---|
input_dataset
|
A Tensor of type variant.
A variant tensor representing the input dataset.
|
path
|
A Tensor of type string.
The path we should write snapshots to / read snapshots from.
|
reader_func_other_args
|
A list of Tensor objects.
|
shard_func_other_args
|
A list of Tensor objects.
|
output_types
|
A list of tf.DTypes that has length >= 1.
|
output_shapes
|
A list of shapes (each a tf.TensorShape or list of ints) that has length >= 1.
|
reader_func
|
A function decorated with @Defun.
Optional. A function to control how to read data from snapshot shards.
|
shard_func
|
A function decorated with @Defun.
Optional. A function to control how to shard data when writing a snapshot.
|
compression
|
An optional string. Defaults to "".
The type of compression to be applied to the saved snapshot files.
|
reader_prefix
|
An optional string. Defaults to "".
|
writer_prefix
|
An optional string. Defaults to "".
|
hash_valid
|
An optional bool. Defaults to False.
|
hash
|
An optional int. Defaults to 0.
|
metadata
|
An optional string. Defaults to "".
|
name
|
A name for the operation (optional).
|
Returns |
|---|
A Tensor of type variant.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates. Some content is licensed under the numpy license.
Last updated 2024-04-26 UTC.
[null,null,["Last updated 2024-04-26 UTC."],[],[],null,["# tf.raw_ops.SnapshotDatasetV2\n\n\u003cbr /\u003e\n\nCreates a dataset that will write to / read from a snapshot.\n\n#### View aliases\n\n\n**Compat aliases for migration**\n\nSee\n[Migration guide](https://www.tensorflow.org/guide/migrate) for\nmore details.\n\n[`tf.compat.v1.raw_ops.SnapshotDatasetV2`](https://www.tensorflow.org/api_docs/python/tf/raw_ops/SnapshotDatasetV2)\n\n\u003cbr /\u003e\n\n tf.raw_ops.SnapshotDatasetV2(\n input_dataset,\n path,\n reader_func_other_args,\n shard_func_other_args,\n output_types,\n output_shapes,\n reader_func,\n shard_func,\n compression='',\n reader_prefix='',\n writer_prefix='',\n hash_valid=False,\n hash=0,\n metadata='',\n name=None\n )\n\nThis dataset attempts to determine whether a valid snapshot exists at the\n`snapshot_path`, and reads from the snapshot in lieu of using `input_dataset`.\nIf not, it will run the preprocessing pipeline as usual, and write out a\nsnapshot of the data processed for future use.\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Args ---- ||\n|--------------------------|--------------------------------------------------------------------------------------------------------------|\n| `input_dataset` | A `Tensor` of type `variant`. A variant tensor representing the input dataset. |\n| `path` | A `Tensor` of type `string`. The path we should write snapshots to / read snapshots from. |\n| `reader_func_other_args` | A list of `Tensor` objects. |\n| `shard_func_other_args` | A list of `Tensor` objects. |\n| `output_types` | A list of `tf.DTypes` that has length `\u003e= 1`. |\n| `output_shapes` | A list of shapes (each a [`tf.TensorShape`](../../tf/TensorShape) or list of `ints`) that has length `\u003e= 1`. |\n| `reader_func` | A function decorated with @Defun. Optional. A function to control how to read data from snapshot shards. |\n| `shard_func` | A function decorated with @Defun. Optional. A function to control how to shard data when writing a snapshot. |\n| `compression` | An optional `string`. Defaults to `\"\"`. The type of compression to be applied to the saved snapshot files. |\n| `reader_prefix` | An optional `string`. Defaults to `\"\"`. |\n| `writer_prefix` | An optional `string`. Defaults to `\"\"`. |\n| `hash_valid` | An optional `bool`. Defaults to `False`. |\n| `hash` | An optional `int`. Defaults to `0`. |\n| `metadata` | An optional `string`. Defaults to `\"\"`. |\n| `name` | A name for the operation (optional). |\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n\u003cbr /\u003e\n\n| Returns ------- ||\n|---|---|\n| A `Tensor` of type `variant`. ||\n\n\u003cbr /\u003e"]]
