tff.learning.optimizers.build_adagrad

Returns a tff.learning.optimizers.Optimizer for Adagrad.

The Adagrad optimizer is based on Adaptive Subgradient Methods for Online Learning and Stochastic Optimization

The update rule given learning rate lr, epsilon eps, preconditioner s, weights w and gradients g is:

s = s + g**2
w = w - lr * g / sqrt(s + eps)

learning_rate A positive float for learning rate.
initial_preconditioner_value A non-negative float, initial value for the preconditioner.
epsilon A small non-negative float, used to maintain numerical stability.