tff.learning.optimizers.build_adafactor
Builds an Adafactor optimizer.
tff.learning.optimizers.build_adafactor(
learning_rate: optimizer.Float,
*,
beta_2_decay: optimizer.Float = -0.8,
epsilon_1: optimizer.Float = 1e-30,
epsilon_2: optimizer.Float = 0.001,
clip_threshold: optimizer.Float = 1.0,
relative_step: bool = True
) -> tff.learning.optimizers.Optimizer
An implementation of Adafactor from Shazeer, Noam et al described in
https://arxiv.org/abs/1804.04235
Args |
learning_rate
|
Initial value of the learning rate.
|
beta_2_decay
|
The decay rate of beta_2 .
|
epsilon_1
|
A small offset to keep denomiantor away from zero.
|
epsilon_2
|
A small offset to avoid learning rate becoming two small over
time.
|
clip_threshold
|
The clipping threshold of the Adafactor algorithm.
|
relative_step
|
If True , learning rate is adjusted based on number of
iterations. This is the default Adafactor learning rate decay.
|
Except as otherwise noted, the content of this page is licensed under the Creative Commons Attribution 4.0 License, and code samples are licensed under the Apache 2.0 License. For details, see the Google Developers Site Policies. Java is a registered trademark of Oracle and/or its affiliates.
Last updated 2024-09-20 UTC.
[null,null,["Last updated 2024-09-20 UTC."],[],[]]