tf.linalg.LinearOperatorCirculant

TensorFlow 1 version

View source on GitHub

LinearOperator acting like a circulant matrix.

View aliases

Compat aliases for migration

See Migration guide for more details.

tf.compat.v1.linalg.LinearOperatorCirculant

tf.linalg.LinearOperatorCirculant(
    spectrum, input_output_dtype=tf.dtypes.complex64, is_non_singular=None,
    is_self_adjoint=None, is_positive_definite=None, is_square=True,
    name='LinearOperatorCirculant'
)

This operator acts like a circulant matrix A with shape [B1,...,Bb, N, N] for some b >= 0. The first b indices index a batch member. For every batch index (i1,...,ib), A[i1,...,ib, : :] is an N x N matrix. This matrix A is not materialized, but for purposes of broadcasting this shape will be relevant.

Description in terms of circulant matrices

Circulant means the entries of A are generated by a single vector, the convolution kernel h: A_{mn} := h_{m-n mod N}. With h = [w, x, y, z],

A = |w z y x|
    |x w z y|
    |y x w z|
    |z y x w|

This means that the result of matrix multiplication v = Au has Lth column given circular convolution between h with the Lth column of u.

Description in terms of the frequency spectrum

There is an equivalent description in terms of the [batch] spectrum H and Fourier transforms. Here we consider A.shape = [N, N] and ignore batch dimensions. Define the discrete Fourier transform (DFT) and its inverse by

DFT[ h[n] ] = H[k] := sum_{n = 0}^{N - 1} h_n e^{-i 2pi k n / N}
IDFT[ H[k] ] = h[n] = N^{-1} sum_{k = 0}^{N - 1} H_k e^{i 2pi k n / N}

From these definitions, we see that

H[0] = sum_{n = 0}^{N - 1} h_n
H[1] = "the first positive frequency"
H[N - 1] = "the first negative frequency"

Loosely speaking, with * element-wise multiplication, matrix multiplication is equal to the action of a Fourier multiplier: A u = IDFT[ H * DFT[u] ]. Precisely speaking, given [N, R] matrix u, let DFT[u] be the [N, R] matrix with rth column equal to the DFT of the rth column of u. Define the IDFT similarly. Matrix multiplication may be expressed columnwise:

Operator properties deduced from the spectrum.

Letting U be the kth Euclidean basis vector, and U = IDFT[u]. The above formulas show thatA U = H_k * U. We conclude that the elements of H are the eigenvalues of this operator. Therefore

This operator is positive definite if and only if Real{H} > 0.

A general property of Fourier transforms is the correspondence between Hermitian functions and real valued transforms.

Suppose H.shape = [B1,...,Bb, N]. We say that H is a Hermitian spectrum if, with % meaning modulus division,

This operator corresponds to a real matrix if and only if H is Hermitian.
This operator is self-adjoint if and only if H is real.

See e.g. "Discrete-Time Signal Processing", Oppenheim and Schafer.

Example of a self-adjoint positive definite operator

# spectrum is real ==> operator is self-adjoint
# spectrum is positive ==> operator is positive definite
spectrum = [6., 4, 2]

operator = LinearOperatorCirculant(spectrum)

# IFFT[spectrum]
operator.convolution_kernel()
==> [4 + 0j, 1 + 0.58j, 1 - 0.58j]

operator.to_dense()
==> [[4 + 0.0j, 1 - 0.6j, 1 + 0.6j],
     [1 + 0.6j, 4 + 0.0j, 1 - 0.6j],
     [1 - 0.6j, 1 + 0.6j, 4 + 0.0j]]

Example of defining in terms of a real convolution kernel

# convolution_kernel is real ==> spectrum is Hermitian.
convolution_kernel = [1., 2., 1.]]
spectrum = tf.signal.fft(tf.cast(convolution_kernel, tf.complex64))

# spectrum is Hermitian ==> operator is real.
# spectrum is shape [3] ==> operator is shape [3, 3]
# We force the input/output type to be real, which allows this to operate
# like a real matrix.
operator = LinearOperatorCirculant(spectrum, input_output_dtype=tf.float32)

operator.to_dense()
==> [[ 1, 1, 2],
     [ 2, 1, 1],
     [ 1, 2, 1]]

Example of Hermitian spectrum

# spectrum is shape [3] ==> operator is shape [3, 3]
# spectrum is Hermitian ==> operator is real.
spectrum = [1, 1j, -1j]

operator = LinearOperatorCirculant(spectrum)

operator.to_dense()
==> [[ 0.33 + 0j,  0.91 + 0j, -0.24 + 0j],
     [-0.24 + 0j,  0.33 + 0j,  0.91 + 0j],
     [ 0.91 + 0j, -0.24 + 0j,  0.33 + 0j]

Example of forcing real `dtype` when spectrum is Hermitian

# spectrum is shape [4] ==> operator is shape [4, 4]
# spectrum is real ==> operator is self-adjoint
# spectrum is Hermitian ==> operator is real
# spectrum has positive real part ==> operator is positive-definite.
spectrum = [6., 4, 2, 4]

# Force the input dtype to be float32.
# Cast the output to float32.  This is fine because the operator will be
# real due to Hermitian spectrum.
operator = LinearOperatorCirculant(spectrum, input_output_dtype=tf.float32)

operator.shape
==> [4, 4]

operator.to_dense()
==> [[4, 1, 0, 1],
     [1, 4, 1, 0],
     [0, 1, 4, 1],
     [1, 0, 1, 4]]

# convolution_kernel = tf.signal.ifft(spectrum)
operator.convolution_kernel()
==> [4, 1, 0, 1]

Performance

Suppose operator is a LinearOperatorCirculant of shape [N, N], and x.shape = [N, R]. Then

operator.matmul(x) is O(R*N*Log[N])
operator.solve(x) is O(R*N*Log[N])
operator.determinant() involves a size N reduce_prod.

If instead operator and x have shape [B1,...,Bb, N, N] and [B1,...,Bb, N, R], every operation increases in complexity by B1*...*Bb.

Matrix property hints

This LinearOperator is initialized with boolean flags of the form is_X, for X = non_singular, self_adjoint, positive_definite, square. These have the following meaning:

If is_X == True, callers should expect the operator to have the property X. This is a promise that should be fulfilled, but is not a runtime assert. For example, finite floating point precision may result in these promises being violated.
If is_X == False, callers should expect the operator to not have X.
If is_X == None (the default), callers should have no expectation either way.

References:

Toeplitz and Circulant Matrices - A Review: Gray, 2006 (pdf)

Args
`spectrum`	Shape `[B1,...,Bb, N]` `Tensor`. Allowed dtypes: `float16`, `float32`, `float64`, `complex64`, `complex128`. Type can be different than `input_output_dtype`
`input_output_dtype`	`dtype` for input/output.
`is_non_singular`	Expect that this operator is non-singular.
`is_self_adjoint`	Expect that this operator is equal to its hermitian transpose. If `spectrum` is real, this will always be true.
`is_positive_definite`	Expect that this operator is positive definite, meaning the quadratic form `x^H A x` has positive real part for all nonzero `x`. Note that we do not require the operator to be self-adjoint to be positive-definite. See: https://en.wikipedia.org/wiki/Positive-definite_matrix\ Extension_for_non_symmetric_matrices
`is_square`	Expect that this operator acts like square [batch] matrices.
`name`	A name to prepend to all ops created by this class.

Attributes
`H`	Returns the adjoint of the current `LinearOperator`. Given `A` representing this `LinearOperator`, return `A*`. Note that calling `self.adjoint()` and `self.H` are equivalent.
`batch_shape`	`TensorShape` of batch dimensions of this `LinearOperator`. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `TensorShape([B1,...,Bb])`, equivalent to `A.shape[:-2]`
`block_depth`	Depth of recursively defined circulant blocks defining this `Operator`. With `A` the dense representation of this `Operator`, `block_depth = 1` means `A` is symmetric circulant. For example, `A = \|w z y x\| \|x w z y\| \|y x w z\| \|z y x w\|` `block_depth = 2` means `A` is block symmetric circulant with symmetric circulant blocks. For example, with `W`, `X`, `Y`, `Z` symmetric circulant, `A = \|W Z Y X\| \|X W Z Y\| \|Y X W Z\| \|Z Y X W\|` `block_depth = 3` means `A` is block symmetric circulant with block symmetric circulant blocks.
`block_shape`
`domain_dimension`	Dimension (in the sense of vector spaces) of the domain of this operator. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `N`.
`dtype`	The `DType` of `Tensor`s handled by this `LinearOperator`.
`graph_parents`	List of graph dependencies of this `LinearOperator`. (deprecated) Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Do not call `graph_parents`.
`is_non_singular`
`is_positive_definite`
`is_self_adjoint`
`is_square`	Return `True/False` depending on if this operator is square.
`range_dimension`	Dimension (in the sense of vector spaces) of the range of this operator. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `M`.
`shape`	`TensorShape` of this `LinearOperator`. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `TensorShape([B1,...,Bb, M, N])`, equivalent to `A.shape`.
`spectrum`
`tensor_rank`	Rank (in the sense of tensors) of matrix corresponding to this operator. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `b + 2`.

Args
`x`	`Tensor` with same `dtype` and shape broadcastable to `self.shape`.
`name`	A name to give this `Op`.

Args
`x`	`LinearOperator` or `Tensor` with compatible shape and same `dtype` as `self`. See class docstring for definition of compatibility.
`adjoint`	Python `bool`. If `True`, left multiply by the adjoint: `A^H x`.
`adjoint_arg`	Python `bool`. If `True`, compute `A x^H` where `x^H` is the hermitian transpose (transposition and complex conjugation).
`name`	A name for this `Op`.

Args
`x`	`Tensor` with compatible shape and same `dtype` as `self`. `x` is treated as a [batch] vector meaning for every set of leading dimensions, the last dimension defines a vector. See class docstring for definition of compatibility.
`adjoint`	Python `bool`. If `True`, left multiply by the adjoint: `A^H x`.
`name`	A name for this `Op`.

Args
`rhs`	`Tensor` with same `dtype` as this operator and compatible shape. `rhs` is treated like a [batch] matrix meaning for every set of leading dimensions, the last two dimensions defines a matrix. See class docstring for definition of compatibility.
`adjoint`	Python `bool`. If `True`, solve the system involving the adjoint of this `LinearOperator`: `A^H X = rhs`.
`adjoint_arg`	Python `bool`. If `True`, solve `A X = rhs^H` where `rhs^H` is the hermitian transpose (transposition and complex conjugation).
`name`	A name scope to use for ops added by this method.

tf.linalg.LinearOperatorCirculant

View aliases

Description in terms of circulant matrices

Description in terms of the frequency spectrum

Operator properties deduced from the spectrum.

Example of a self-adjoint positive definite operator

Example of defining in terms of a real convolution kernel

Example of Hermitian spectrum

Example of forcing real dtype when spectrum is Hermitian

Performance

Matrix property hints

References:

Args

Extension_for_non_symmetric_matrices

Attributes

Methods

add_to_tensor

adjoint

assert_hermitian_spectrum

assert_non_singular

assert_positive_definite

assert_self_adjoint

batch_shape_tensor

block_shape_tensor

cholesky

cond

convolution_kernel

determinant

diag_part

domain_dimension_tensor

eigvals

inverse

log_abs_determinant

matmul

matvec

range_dimension_tensor

shape_tensor

solve

Examples:

solvevec

Examples:

Example of forcing real `dtype` when spectrum is Hermitian

`add_to_tensor`

`adjoint`

`assert_hermitian_spectrum`

`assert_non_singular`

`assert_positive_definite`

`assert_self_adjoint`

`batch_shape_tensor`

`block_shape_tensor`

`cholesky`

`cond`

`convolution_kernel`

`determinant`

`diag_part`

`domain_dimension_tensor`

`eigvals`

`inverse`

`log_abs_determinant`

`matmul`

`matvec`

`range_dimension_tensor`

`shape_tensor`

`solve`

`solvevec`