Creates a dataset that shards the input dataset.
Creates a dataset that shards the input dataset by num_workers, returning a sharded dataset for the index-th worker. This attempts to automatically shard a dataset by examining the Dataset graph and inserting a shard op before the inputs to a reader Dataset (e.g. CSVDataset, TFRecordDataset).
This dataset will throw a NotFound error if we cannot shard the dataset automatically.
Nested Classes
class | ExperimentalAutoShardDataset.Options |
Optional attributes for
ExperimentalAutoShardDataset
|
Public Methods
Output <Object> |
asOutput
()
Returns the symbolic handle of a tensor.
|
static ExperimentalAutoShardDataset.Options |
autoShardPolicy
(Long autoShardPolicy)
|
static ExperimentalAutoShardDataset | |
Output <?> |
handle
()
|
Inherited Methods
Public Methods
public Output <Object> asOutput ()
Returns the symbolic handle of a tensor.
Inputs to TensorFlow operations are outputs of another TensorFlow operation. This method is used to obtain a symbolic handle that represents the computation of the input.
public static ExperimentalAutoShardDataset create ( Scope scope, Operand <?> inputDataset, Operand <Long> numWorkers, Operand <Long> index, List<Class<?>> outputTypes, List< Shape > outputShapes, Options... options)
Factory method to create a class wrapping a new ExperimentalAutoShardDataset operation.
Parameters
scope | current scope |
---|---|
inputDataset | A variant tensor representing the input dataset. |
numWorkers | A scalar representing the number of workers to distribute this dataset across. |
index | A scalar representing the index of the current worker out of num_workers. |
options | carries optional attributes values |
Returns
- a new instance of ExperimentalAutoShardDataset