TensorFlow 1 version
View source on GitHub
Base class for cross-device reduction and broadcasting algorithms.
View aliases
Compat aliases for migration
See Migration guide for more details.
tf.distribute.CrossDeviceOps()
Methods
batch_reduce
batch_reduce(
reduce_op, value_destination_pairs
)
Reduce PerReplica objects in a batch.
Reduce each first element in value_destination_pairs to each second
element which indicates the destinations.
This can be faster than multiple individual reduces because we can
fuse several tensors into one or multiple packs before reduction.
| Args | |
|---|---|
reduce_op
|
An instance of tf.distribute.ReduceOp that indicates how
the per_replica_value will be reduced.
|
value_destination_pairs
|
a list or a tuple of PerReplica objects (or tensors with device set if there is one device) and destinations. |
| Returns | |
|---|---|
| a list of Mirrored objects. |
| Raises | |
|---|---|
ValueError
|
if value_destination_pairs is not an iterable of
tuples of PerReplica objects and destinations.
|
batch_reduce_implementation
batch_reduce_implementation(
reduce_op, value_destination_pairs
)
Implementation of reduce PerReplica objects in a batch.
Overriding this method is useful for subclass implementers.
Reduce each first element in value_destination_pairs to each second
element which indicates the destinations.
| Args | |
|---|---|
reduce_op
|
An instance of tf.distribute.ReduceOp that indicates how
per_replica_value will be reduced.
|
value_destination_pairs
|
an iterable of tuples of PerReplica objects (or tensors with device set if there is one device) and destinations. |
| Returns | |
|---|---|
| a list of Mirrored objects. |
| Raises | |
|---|---|
ValueError
|
if value_destination_pairs is not an iterable of
tuples of PerReplica objects and destinations
|
broadcast
broadcast(
tensor, destinations
)
Broadcast the tensor to destinations.
| Args | |
|---|---|
tensor
|
the tensor to broadcast. |
destinations
|
the broadcast destinations. |
| Returns | |
|---|---|
| a Mirrored object. |
broadcast_implementation
broadcast_implementation(
tensor, destinations
)
Implementation of broadcast the tensor to destinations.
| Args | |
|---|---|
tensor
|
the tensor to broadcast. |
destinations
|
the broadcast destinations. |
| Returns | |
|---|---|
| a Mirrored object. |
reduce
reduce(
reduce_op, per_replica_value, destinations
)
Reduce per_replica_value to destinations.
It runs the reduction operation defined by reduce_op and put the
result on destinations.
| Args | |
|---|---|
reduce_op
|
An instance of tf.distribute.ReduceOp that indicates how
per_replica_value will be reduced.
|
per_replica_value
|
a PerReplica object or a tensor with device set. |
destinations
|
the reduction destinations. |
| Returns | |
|---|---|
| a Mirrored object. |
| Raises | |
|---|---|
ValueError
|
if per_replica_value can't be converted to a PerReplica object or if destinations aren't strings, Variables or DistributedValues |
reduce_implementation
reduce_implementation(
reduce_op, per_replica_value, destinations
)
The implementation of reduce of per_replica_value to destinations.
Overriding this method is useful for subclass implementers.
It runs the reduction operation defined by reduce_op and put the
result on destinations.
| Args | |
|---|---|
reduce_op
|
An instance tf.distribute.ReduceOp that indicates of how
per_replica_value will be reduced.
|
per_replica_value
|
a PerReplica object or a tensor with device set. |
destinations
|
the reduction destinations. |
| Returns | |
|---|---|
| a Mirrored object. |
| Raises | |
|---|---|
ValueError
|
if per_replica_value can't be converted to a PerReplica object. |