Module: tfm.optimization.lr_cfg

Dataclasses for learning rate schedule config.

Classes

class ConstantLrConfig: Configuration for constant learning rate.

class CosineLrConfig: Configuration for Cosine learning rate decay.

class DirectPowerLrConfig: Configuration for DirectPower learning rate decay.

class ExponentialLrConfig: Configuration for exponential learning rate decay.

class LinearWarmupConfig: Configuration for linear warmup schedule config.

class PolynomialLrConfig: Configuration for polynomial learning rate decay.

class PolynomialWarmupConfig: Configuration for linear warmup schedule config.

class PowerAndLinearDecayLrConfig: Configuration for DirectPower learning rate decay.

class PowerDecayWithOffsetLrConfig: Configuration for power learning rate decay with step offset.

class StepCosineLrConfig: Configuration for stepwise learning rate decay.

class StepwiseLrConfig: Configuration for stepwise learning rate decay.