tfa.optimizers.CyclicalLearningRate

A LearningRateSchedule that uses cyclical schedule.

tfa.optimizers.CyclicalLearningRate(
    initial_learning_rate: Union[FloatTensorLike, Callable],
    maximal_learning_rate: Union[FloatTensorLike, Callable],
    step_size: tfa.types.FloatTensorLike,
    scale_fn: Callable,
    scale_mode: str = 'cycle',
    name: str = 'CyclicalLearningRate'
)

Used in the notebooks

Used in the tutorials

initial_learning_rate A scalar float32 or float64 Tensor or a Python number. The initial learning rate.
maximal_learning_rate A scalar float32 or float64 Tensor or a Python number. The maximum learning rate.
step_size A scalar float32 or float64 Tensor or a Python number. Step size denotes the number of training iterations it takes to get to maximal_learning_rate.
scale_fn A function. Scheduling function applied in cycle
scale_mode ['cycle', 'iterations']. Mode to apply during cyclic schedule
name (Optional) Name for the operation.

Methods

from_config

@classmethod
from_config(
    config
)

Instantiates a LearningRateSchedule from its config.

Args
config Output of get_config().

Returns
A LearningRateSchedule instance.

get_config

View source

get_config()

__call__

View source

__call__(
    step
)

Call self as a function.