Types of loss reduction.
Contains the following values:
AUTO: Indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to
SUM_OVER_BATCH_SIZE. When used with
tf.distribute.Strategy, outside of built-in training loops such as
fit, we expect reduction value to be
AUTOin that case will raise an error.
NONE: Un-reduced weighted losses with the same shape as input. When this reduction type used with built-in Keras training loops like
evaluate, the unreduced vector loss is passed to the optimizer but the reported loss will be a scalar value.
SUM: Scalar sum of weighted losses.
SUMdivided by number of elements in losses. This reduction type is not supported when used with
tf.distribute.Strategyoutside of built-in training loops like
You can implement 'SUM_OVER_BATCH_SIZE' using global batch size like:
with strategy.scope(): loss_obj = tf.keras.losses.CategoricalCrossentropy( reduction=tf.keras.losses.Reduction.None) .... loss = tf.reduce_sum(loss_object(labels, predictions)) * (1. / global_batch_size)
Please see https://www.tensorflow.org/alpha/tutorials/distribute/training_loops for more details on this.
@classmethod validate( cls, key )