![]() |
Returns a tff.learning.optimizers.Optimizer
for Adagrad.
tff.learning.optimizers.build_adagrad(
learning_rate: float,
initial_preconditioner_value: float = 0.1,
epsilon: float = 1e-07
) -> tff.learning.optimizers.Optimizer
The Adagrad optimizer is based on Adaptive Subgradient Methods for Online Learning and Stochastic Optimization
The update rule given learning rate lr
, epsilon eps
, preconditioner s
,
weights w
and gradients g
is:
s = s + g**2
w = w - lr * g / sqrt(s + eps)
Args | |
---|---|
learning_rate
|
A positive float for learning rate. |
initial_preconditioner_value
|
A non-negative float, initial value for the preconditioner. |
epsilon
|
A small non-negative float, used to maintain numerical stability. |