tff.learning.optimizers.build_adafactor

Builds an Adafactor optimizer.

tff.learning.optimizers.build_adafactor(
    learning_rate: optimizer.Float,
    *,
    beta_2_decay: optimizer.Float = -0.8,
    epsilon_1: optimizer.Float = 1e-30,
    epsilon_2: optimizer.Float = 0.001,
    clip_threshold: optimizer.Float = 1.0,
    relative_step: bool = True
) -> tff.learning.optimizers.Optimizer

An implementation of Adafactor from Shazeer, Noam et al described in https://arxiv.org/abs/1804.04235

Args
`learning_rate`	Initial value of the learning rate.
`beta_2_decay`	The decay rate of `beta_2`.
`epsilon_1`	A small offset to keep denomiantor away from zero.
`epsilon_2`	A small offset to avoid learning rate becoming two small over time.
`clip_threshold`	The clipping threshold of the Adafactor algorithm.
`relative_step`	If `True`, learning rate is adjusted based on number of iterations. This is the default Adafactor learning rate decay.

Returns
A `tff.learning.optimizers.Optimizer` that implements the Adafactor optimizer.

tff.learning.optimizers.build_adafactor

Args

Returns