tf_agents.utils.common.ornstein_uhlenbeck_process

An op for generating noise from a zero-mean Ornstein-Uhlenbeck process.

tf_agents.utils.common.ornstein_uhlenbeck_process(
    initial_value, damping=0.15, stddev=0.2, seed=None,
    scope='ornstein_uhlenbeck_noise'
)

The Ornstein-Uhlenbeck process is a process that generates temporally correlated noise via a random walk with damping. This process describes the velocity of a particle undergoing brownian motion in the presence of friction. This can be useful for exploration in continuous action environments with momentum.

The temporal update equation is: x_next = (1 - damping) * x + N(0, std_dev)

initial_value Initial value of the process.
damping The rate at which the noise trajectory is damped towards the mean. We must have 0 <= damping <= 1, where a value of 0 gives an undamped random walk and a value of 1 gives uncorrelated Gaussian noise. Hence in most applications a small non-zero value is appropriate.
stddev Standard deviation of the Gaussian component.
seed Seed for random number generation.
scope Scope of the variables.

An op that generates noise.