Join us at TensorFlow World, Oct 28-31. Use code TF20 for 20% off select passes. Register now

tf.train.experimental.FixedLossScale

TensorFlow 2.0 version View source on GitHub

Class FixedLossScale

Loss scale with a fixed value.

Inherits From: LossScale

Aliases:

  • Class tf.compat.v1.train.experimental.FixedLossScale
  • Class tf.compat.v2.train.experimental.FixedLossScale

The loss scale is not updated for the lifetime of instances of this class. A given instance of this class always returns the same number when called.

__init__

View source

__init__(loss_scale_value)

Creates the fixed loss scale.

Args:

  • loss_scale_value: A Python float. Its ideal value varies depending on models to run. Choosing a too small loss_scale might affect model quality; a too big loss_scale might cause inf or nan. There is no single right loss_scale to apply. There is no harm choosing a relatively big number as long as no nan or inf is encountered in training.

Raises:

  • ValueError: If loss_scale is less than 1.

Methods

__call__

View source

__call__()

from_config

View source

from_config(
    cls,
    config
)

Creates the LossScale from its config.

get_config

View source

get_config()

update

View source

update(grads)