View source on GitHub
|
Windowed estimates of variance.
tfp.stats.windowed_variance(
x, low_indices=None, high_indices=None, axis=0, name=None
)
Computes variances among data in the Tensor x along the given windows:
result[i] = variance(x[low_indices[i]:high_indices[i]+1])
accurately and efficiently. To wit, if K is the size of
low_indices and high_indices, and N is the size of x along
the given axis, the computation takes O(K + N) work, O(log(N))
depth (the length of the longest series of operations that are
performed sequentially), and only uses O(1) TensorFlow kernel
invocations. The underlying algorithm is an adaptation of the
streaming reduction for accurate variance computations given in [1].
This function can be useful for assessing the behavior over time of trailing-window estimators from some iterative process, such as the last half of an MCMC chain.
Suppose x has shape Bx + [N] + E, where the Bx component has
rank axis, and low_indices and high_indices broadcast to shape
[M]. Then each element of low_indices and high_indices
must be between 0 and N+1, and the shape of the output will be
Bx + [M] + E. Batch shape in the indices is not currently supported.
The default windows are
[0, 1), [1, 2), [1, 3), [2, 4), [2, 5), ...
This corresponds to analyzing x as though it were streaming, for
example successive states of an MCMC sampler, and we were interested
in the variance of the last half of the data at each point.
Args | |
|---|---|
x
|
A numeric Tensor holding N samples along the given axis,
whose windowed variances are desired.
|
low_indices
|
An integer Tensor defining the lower boundary
(inclusive) of each window. Default: elementwise half of
high_indices.
|
high_indices
|
An integer Tensor defining the upper boundary
(exclusive) of each window. Must be broadcast-compatible with
low_indices. Default: tf.range(1, N+1), i.e., N windows
that each end in the corresponding datum from x (inclusive).
</td>
</tr><tr>
<td>axis</td>
<td>
ScalarTensordesignating the axis holding samples. This
is the axis ofxalong which we take windows, and therefore
the axis thatlow_indicesandhigh_indicesindex into.
Other axes are treated in batch. Default value:0(leftmost
dimension).
</td>
</tr><tr>
<td>name</td>
<td>
Pythonstrname prefixed to Ops created by this function.
Default value:None(i.e.,'windowed_variance'`).
|
Returns | |
|---|---|
variances
|
A numeric Tensor holding the windowed variances of
x along the axis dimension.
|
References
[1]: Philippe Pebay. Formulas for Robust, One-Pass Parallel Computation of Covariances and Arbitrary-Order Statistical Moments. Technical Report SAND2008-6212, 2008. https://prod-ng.sandia.gov/techlib-noauth/access-control.cgi/2008/086212.pdf