tf.data.experimental.save

<!-- Stable --> <table class="tfo-notebook-buttons tfo-api nocontent" align="left"> <td> <a target="_blank" href="https://github.com/tensorflow/tensorflow/blob/v2.5.0/tensorflow/python/data/experimental/ops/io.py#L41-L117"> <img src="https://www.tensorflow.org/images/GitHub-Mark-32px.png" /> View source on GitHub </a> </td> </table> Saves the content of the given dataset. <pre class="devsite-click-to-copy prettyprint lang-py tfo-signature-link"> <code>tf.data.experimental.save( dataset, path, compression=None, shard_func=None ) </code></pre> <!-- Placeholder for "Used in" --> #### Example usage: <pre class="devsite-click-to-copy prettyprint lang-py"> <code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;">import tempfile</code> <code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;">path = os.path.join(tempfile.gettempdir(), &quot;saved_data&quot;)</code> <code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;"># Save a dataset</code> <code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;">dataset = tf.data.Dataset.range(2)</code> <code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;">tf.data.experimental.save(dataset, path)</code> <code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;">new_dataset = tf.data.experimental.load(path)</code> <code class="devsite-terminal" data-terminal-prefix="&gt;&gt;&gt;">for elem in new_dataset:</code> <code class="devsite-terminal" data-terminal-prefix="..."> print(elem)</code> <code class="no-select nocode">tf.Tensor(0, shape=(), dtype=int64)</code> <code class="no-select nocode">tf.Tensor(1, shape=(), dtype=int64)</code> </pre> The saved dataset is saved in multiple file "shards". By default, the dataset output is divided to shards in a round-robin fashion but custom sharding can be specified via the `shard_func` function. For example, you can save the dataset to using a single shard as follows: ```python dataset = make_dataset() def custom_shard_func(element): return 0 dataset = tf.data.experimental.save( path="/path/to/data", ..., shard_func=custom_shard_func)

dataset The dataset to save.
path Required. A directory to use for saving the dataset.
compression Optional. The algorithm to use to compress data when writing it. Supported options are GZIP and NONE. Defaults to NONE.
shard_func Optional. A function to control the mapping of dataset elements to file shards. The function is expected to map elements of the input dataset to int64 shard IDs. If present, the function will be traced and executed as graph computation.