ExperimentalAutoShardDataset

public final class ExperimentalAutoShardDataset

Creates a dataset that shards the input dataset.

Creates a dataset that shards the input dataset by num_workers, returning a sharded dataset for the index-th worker. This attempts to automatically shard a dataset by examining the Dataset graph and inserting a shard op before the inputs to a reader Dataset (e.g. CSVDataset, TFRecordDataset).

This dataset will throw a NotFound error if we cannot shard the dataset automatically.

Nested Classes

class ExperimentalAutoShardDataset.Options Optional attributes for ExperimentalAutoShardDataset  

Public Methods

Output<Object>
asOutput()
Returns the symbolic handle of a tensor.
static ExperimentalAutoShardDataset.Options
autoShardPolicy(Long autoShardPolicy)
static ExperimentalAutoShardDataset
create(Scope scope, Operand<?> inputDataset, Operand<Long> numWorkers, Operand<Long> index, List<Class<?>> outputTypes, List<Shape> outputShapes, Options... options)
Factory method to create a class wrapping a new ExperimentalAutoShardDataset operation.
Output<?>
handle()

Inherited Methods

org.tensorflow.op.PrimitiveOp
final boolean
equals(Object obj)
final int
hashCode()
Operation
op()
Returns the underlying Operation
final String
toString()
boolean
equals(Object arg0)
final Class<?>
getClass()
int
hashCode()
final void
notify()
final void
notifyAll()
String
toString()
final void
wait(long arg0, int arg1)
final void
wait(long arg0)
final void
wait()
org.tensorflow.Operand
abstract Output<Object>
asOutput()
Returns the symbolic handle of a tensor.

Public Methods

public Output<Object> asOutput ()

Returns the symbolic handle of a tensor.

Inputs to TensorFlow operations are outputs of another TensorFlow operation. This method is used to obtain a symbolic handle that represents the computation of the input.

public static ExperimentalAutoShardDataset.Options autoShardPolicy (Long autoShardPolicy)

public static ExperimentalAutoShardDataset create (Scope scope, Operand<?> inputDataset, Operand<Long> numWorkers, Operand<Long> index, List<Class<?>> outputTypes, List<Shape> outputShapes, Options... options)

Factory method to create a class wrapping a new ExperimentalAutoShardDataset operation.

Parameters
scope current scope
inputDataset A variant tensor representing the input dataset.
numWorkers A scalar representing the number of workers to distribute this dataset across.
index A scalar representing the index of the current worker out of num_workers.
options carries optional attributes values
Returns
  • a new instance of ExperimentalAutoShardDataset

public Output<?> handle ()