tf.keras.losses.Reduction

View source on GitHub

Types of loss reduction.

Contains the following values:

  • AUTO: Indicates that the reduction option will be determined by the usage context. For almost all cases this defaults to SUM_OVER_BATCH_SIZE. When used with tf.distribute.Strategy, outside of built-in training loops such as tf.keras compile and fit, we expect reduction value to be SUM or NONE. Using AUTO in that case will raise an error.
  • NONE: Weighted losses with one dimension reduced (axis=-1, or axis specified by loss function). When this reduction type used with built-in Keras training loops like fit/evaluate, the unreduced vector loss is passed to the optimizer but the reported loss will be a scalar value.
  • SUM: Scalar sum of weighted losses.

  • SUM_OVER_BATCH_SIZE: Scalar SUM divided by number of elements in losses. This reduction type is not supported when used with tf.distribute.Strategy outside of built-in training loops like tf.keras compile/fit.

    You can implement 'SUM_OVER_BATCH_SIZE' using global batch size like:

with strategy.scope():
  loss_obj = tf.keras.losses.CategoricalCrossentropy(
      reduction=tf.keras.losses.Reduction.NONE)
  ....
  loss = tf.reduce_sum(loss_object(labels, predictions)) *
      (1. / global_batch_size)

Please see https://www.tensorflow.org/alpha/tutorials/distribute/training_loops for more details on this.

Methods

all

View source

@classmethod
all()

validate

View source

@classmethod
validate(
    key
)

Class Variables

  • AUTO = 'auto'
  • NONE = 'none'
  • SUM = 'sum'
  • SUM_OVER_BATCH_SIZE = 'sum_over_batch_size'