tf_agents.replay_buffers.reverb_utils.ReverbAddTrajectoryObserver

Stateful observer for writing to the Reverb replay.

tf_agents.replay_buffers.reverb_utils.ReverbAddTrajectoryObserver(
    py_client: tf_agents.typing.types.ReverbClient,
    table_name: Text,
    sequence_length: int,
    stride_length: int = 1,
    priority: Union[float, int] = 1
)

Used in the notebooks

Used in the tutorials
SAC minitaur with the Actor-Learner API

Args
py_client	Python client for the reverb replay server.
table_name	The table name where samples will be written to.
sequence_length	The sequence_length used to write to the given table.
stride_length	The integer stride for the sliding window for overlapping sequences. The default value of 1 creates an item for every window. Using L = sequence_length this means items are created for times {0, 1, .., L-1}, {1, 2, .., L}, .... In contrast, stride_length = L will create an item only for disjoint windows {0, 1, ..., L-1}, {L, ..., 2 * L - 1}, ....
priority	Initial priority for new samples in the RB.

Raises
ValueError	If table_names or sequence_lengths are not lists or their lengths are not equal.

Methods

close

View source

close()

Closes the writer of the observer.

Note: Using the observer after closing it is not supported.

open

View source

open()

Open the writer of the observer.

reset

View source

reset()

Resets the state of the observer.

No data observed before the reset will be pushed to the RB.

__call__

View source

__call__(
    trajectory
)

Writes the trajectory into the underlying replay buffer.

Allows trajectory to be a flattened trajectory. No batch dimension allowed.

Args
trajectory	The trajectory to be written which could be (possibly nested) trajectory object or a flattened version of a trajectory. It assumes there is no batch dimension.