tfp.substrates.jax.stats.windowed_variance

Windowed estimates of variance.

tfp.substrates.jax.stats.windowed_variance(
    x, low_indices=None, high_indices=None, axis=0, name=None
)

Computes variances among data in the Tensor x along the given windows:

result[i] = variance(x[low_indices[i]:high_indices[i]+1])

accurately and efficiently. To wit, if K is the size of low_indices and high_indices, and N is the size of x along the given axis, the computation takes O(K + N) work, O(log(N)) depth (the length of the longest series of operations that are performed sequentially), and only uses O(1) TensorFlow kernel invocations. The underlying algorithm is an adaptation of the streaming reduction for accurate variance computations given in [1].

This function can be useful for assessing the behavior over time of trailing-window estimators from some iterative process, such as the last half of an MCMC chain.

Suppose x has shape Bx + [N] + E, where the Bx component has rank axis, and low_indices and high_indices broadcast to shape [M]. Then each element of low_indices and high_indices must be between 0 and N+1, and the shape of the output will be Bx + [M] + E. Batch shape in the indices is not currently supported.

The default windows are [0, 1), [1, 2), [1, 3), [2, 4), [2, 5), ... This corresponds to analyzing x as though it were streaming, for example successive states of an MCMC sampler, and we were interested in the variance of the last half of the data at each point.

x A numeric Tensor holding N samples along the given axis, whose windowed variances are desired.
low_indices An integer Tensor defining the lower boundary (inclusive) of each window. Default: elementwise half of high_indices.
high_indices An integer Tensor defining the upper boundary (exclusive) of each window. Must be broadcast-compatible with low_indices. Default: tf.range(1, N+1), i.e., N windows that each end in the corresponding datum from x (inclusive). </td> </tr><tr> <td>axis<a id="axis"></a> </td> <td> ScalarTensordesignating the axis holding samples. This is the axis ofxalong which we take windows, and therefore the axis thatlow_indicesandhigh_indicesindex into. Other axes are treated in batch. Default value:0(leftmost dimension). </td> </tr><tr> <td>name<a id="name"></a> </td> <td> Pythonstrname prefixed to Ops created by this function. Default value:None(i.e.,'windowed_variance'`).

variances A numeric Tensor holding the windowed variances of x along the axis dimension.

References

[1]: Philippe Pebay. Formulas for Robust, One-Pass Parallel Computation of Covariances and Arbitrary-Order Statistical Moments. Technical Report SAND2008-6212, 2008. https://prod-ng.sandia.gov/techlib-noauth/access-control.cgi/2008/086212.pdf