tf.distribute.experimental.CommunicationImplementation

Cross device communication implementation.

Main aliases

tf.distribute.experimental.CollectiveCommunication

Compat aliases for migration

See Migration guide for more details.

tf.compat.v1.distribute.experimental.CollectiveCommunication, tf.compat.v1.distribute.experimental.CommunicationImplementation

  • AUTO: Automatically chosen by Tensorflow.
  • RING: TensorFlow's ring algorithms for all-reduce and all-gather.
  • NCCL: NVIDIA®'s NCCL library. This is now only used for all-reduce on GPUs; all-reduce on CPU, all-gather and broadcast fallbacks to RING.

AUTO <CommunicationImplementation.AUTO: 'AUTO'>
NCCL <CommunicationImplementation.NCCL: 'NCCL'>
RING <CommunicationImplementation.RING: 'RING'>