tf.raw_ops.AutoShardDataset

Creates a dataset that shards the input dataset.

tf.raw_ops.AutoShardDataset(
    input_dataset,
    num_workers,
    index,
    output_types,
    output_shapes,
    auto_shard_policy=0,
    num_replicas=0,
    name=None
)

Creates a dataset that shards the input dataset by num_workers, returning a sharded dataset for the index-th worker. This attempts to automatically shard a dataset by examining the Dataset graph and inserting a shard op before the inputs to a reader Dataset (e.g. CSVDataset, TFRecordDataset).

This dataset will throw a NotFound error if we cannot shard the dataset automatically.

input_dataset A Tensor of type variant. A variant tensor representing the input dataset.
num_workers A Tensor of type int64. A scalar representing the number of workers to distribute this dataset across.
index A Tensor of type int64. A scalar representing the index of the current worker out of num_workers.
output_types A list of tf.DTypes that has length >= 1.
output_shapes A list of shapes (each a tf.TensorShape or list of ints) that has length >= 1.
auto_shard_policy An optional int. Defaults to 0.
num_replicas An optional int. Defaults to 0.
name A name for the operation (optional).

A Tensor of type variant.