tf.linalg.LinearOperatorCirculant

TensorFlow 1 version

View source on GitHub

LinearOperator acting like a circulant matrix.

View aliases

Compat aliases for migration

See Migration guide for more details.

tf.compat.v1.linalg.LinearOperatorCirculant

tf.linalg.LinearOperatorCirculant(
    spectrum, input_output_dtype=tf.dtypes.complex64, is_non_singular=None,
    is_self_adjoint=None, is_positive_definite=None, is_square=True,
    name='LinearOperatorCirculant'
)

This operator acts like a circulant matrix A with shape [B1,...,Bb, N, N] for some b >= 0. The first b indices index a batch member. For every batch index (i1,...,ib), A[i1,...,ib, : :] is an N x N matrix. This matrix A is not materialized, but for purposes of broadcasting this shape will be relevant.

Description in terms of circulant matrices

Circulant means the entries of A are generated by a single vector, the convolution kernel h: A_{mn} := h_{m-n mod N}. With h = [w, x, y, z],

A = |w z y x|
    |x w z y|
    |y x w z|
    |z y x w|

This means that the result of matrix multiplication v = Au has Lth column given circular convolution between h with the Lth column of u.

Description in terms of the frequency spectrum

There is an equivalent description in terms of the [batch] spectrum H and Fourier transforms. Here we consider A.shape = [N, N] and ignore batch dimensions. Define the discrete Fourier transform (DFT) and its inverse by

DFT[ h[n] ] = H[k] := sum_{n = 0}^{N - 1} h_n e^{-i 2pi k n / N}
IDFT[ H[k] ] = h[n] = N^{-1} sum_{k = 0}^{N - 1} H_k e^{i 2pi k n / N}

From these definitions, we see that

H[0] = sum_{n = 0}^{N - 1} h_n
H[1] = "the first positive frequency"
H[N - 1] = "the first negative frequency"

Loosely speaking, with * element-wise multiplication, matrix multiplication is equal to the action of a Fourier multiplier: A u = IDFT[ H * DFT[u] ]. Precisely speaking, given [N, R] matrix u, let DFT[u] be the [N, R] matrix with rth column equal to the DFT of the rth column of u. Define the IDFT similarly. Matrix multiplication may be expressed columnwise:

Operator properties deduced from the spectrum.

Letting U be the kth Euclidean basis vector, and U = IDFT[u]. The above formulas show thatA U = H_k * U. We conclude that the elements of H are the eigenvalues of this operator. Therefore

This operator is positive definite if and only if Real{H} > 0.

A general property of Fourier transforms is the correspondence between Hermitian functions and real valued transforms.

Suppose H.shape = [B1,...,Bb, N]. We say that H is a Hermitian spectrum if, with % meaning modulus division,

This operator corresponds to a real matrix if and only if H is Hermitian.
This operator is self-adjoint if and only if H is real.

See e.g. "Discrete-Time Signal Processing", Oppenheim and Schafer.

Example of a self-adjoint positive definite operator

# spectrum is real ==> operator is self-adjoint
# spectrum is positive ==> operator is positive definite
spectrum = [6., 4, 2]

operator = LinearOperatorCirculant(spectrum)

# IFFT[spectrum]
operator.convolution_kernel()
==> [4 + 0j, 1 + 0.58j, 1 - 0.58j]

operator.to_dense()
==> [[4 + 0.0j, 1 - 0.6j, 1 + 0.6j],
     [1 + 0.6j, 4 + 0.0j, 1 - 0.6j],
     [1 - 0.6j, 1 + 0.6j, 4 + 0.0j]]

Example of defining in terms of a real convolution kernel

# convolution_kernel is real ==> spectrum is Hermitian.
convolution_kernel = [1., 2., 1.]]
spectrum = tf.signal.fft(tf.cast(convolution_kernel, tf.complex64))

# spectrum is Hermitian ==> operator is real.
# spectrum is shape [3] ==> operator is shape [3, 3]
# We force the input/output type to be real, which allows this to operate
# like a real matrix.
operator = LinearOperatorCirculant(spectrum, input_output_dtype=tf.float32)

operator.to_dense()
==> [[ 1, 1, 2],
     [ 2, 1, 1],
     [ 1, 2, 1]]

Example of Hermitian spectrum

# spectrum is shape [3] ==> operator is shape [3, 3]
# spectrum is Hermitian ==> operator is real.
spectrum = [1, 1j, -1j]

operator = LinearOperatorCirculant(spectrum)

operator.to_dense()
==> [[ 0.33 + 0j,  0.91 + 0j, -0.24 + 0j],
     [-0.24 + 0j,  0.33 + 0j,  0.91 + 0j],
     [ 0.91 + 0j, -0.24 + 0j,  0.33 + 0j]

Example of forcing real `dtype` when spectrum is Hermitian

# spectrum is shape [4] ==> operator is shape [4, 4]
# spectrum is real ==> operator is self-adjoint
# spectrum is Hermitian ==> operator is real
# spectrum has positive real part ==> operator is positive-definite.
spectrum = [6., 4, 2, 4]

# Force the input dtype to be float32.
# Cast the output to float32.  This is fine because the operator will be
# real due to Hermitian spectrum.
operator = LinearOperatorCirculant(spectrum, input_output_dtype=tf.float32)

operator.shape
==> [4, 4]

operator.to_dense()
==> [[4, 1, 0, 1],
     [1, 4, 1, 0],
     [0, 1, 4, 1],
     [1, 0, 1, 4]]

# convolution_kernel = tf.signal.ifft(spectrum)
operator.convolution_kernel()
==> [4, 1, 0, 1]

Performance

Suppose operator is a LinearOperatorCirculant of shape [N, N], and x.shape = [N, R]. Then

operator.matmul(x) is O(R*N*Log[N])
operator.solve(x) is O(R*N*Log[N])
operator.determinant() involves a size N reduce_prod.

If instead operator and x have shape [B1,...,Bb, N, N] and [B1,...,Bb, N, R], every operation increases in complexity by B1*...*Bb.

Matrix property hints

This LinearOperator is initialized with boolean flags of the form is_X, for X = non_singular, self_adjoint, positive_definite, square. These have the following meaning:

If is_X == True, callers should expect the operator to have the property X. This is a promise that should be fulfilled, but is not a runtime assert. For example, finite floating point precision may result in these promises being violated.
If is_X == False, callers should expect the operator to not have X.
If is_X == None (the default), callers should have no expectation either way.

References:

Toeplitz and Circulant Matrices - A Review: Gray, 2006 (pdf)

Args
`spectrum`	Shape `[B1,...,Bb, N]` `Tensor`. Allowed dtypes: `float16`, `float32`, `float64`, `complex64`, `complex128`. Type can be different than `input_output_dtype`
`input_output_dtype`	`dtype` for input/output.
`is_non_singular`	Expect that this operator is non-singular.
`is_self_adjoint`	Expect that this operator is equal to its hermitian transpose. If `spectrum` is real, this will always be true.
`is_positive_definite`	Expect that this operator is positive definite, meaning the quadratic form `x^H A x` has positive real part for all nonzero `x`. Note that we do not require the operator to be self-adjoint to be positive-definite. See: https://en.wikipedia.org/wiki/Positive-definite_matrix\ Extension_for_non_symmetric_matrices
`is_square`	Expect that this operator acts like square [batch] matrices.
`name`	A name to prepend to all ops created by this class.

Attributes
`H`	Returns the adjoint of the current `LinearOperator`. Given `A` representing this `LinearOperator`, return `A*`. Note that calling `self.adjoint()` and `self.H` are equivalent.
`batch_shape`	`TensorShape` of batch dimensions of this `LinearOperator`. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `TensorShape([B1,...,Bb])`, equivalent to `A.shape[:-2]`
`block_depth`	Depth of recursively defined circulant blocks defining this `Operator`. With `A` the dense representation of this `Operator`, `block_depth = 1` means `A` is symmetric circulant. For example, `A = \|w z y x\| \|x w z y\| \|y x w z\| \|z y x w\|` `block_depth = 2` means `A` is block symmetric circulant with symmetric circulant blocks. For example, with `W`, `X`, `Y`, `Z` symmetric circulant, `A = \|W Z Y X\| \|X W Z Y\| \|Y X W Z\| \|Z Y X W\|` `block_depth = 3` means `A` is block symmetric circulant with block symmetric circulant blocks.
`block_shape`
`domain_dimension`	Dimension (in the sense of vector spaces) of the domain of this operator. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `N`.
`dtype`	The `DType` of `Tensor`s handled by this `LinearOperator`.
`graph_parents`	List of graph dependencies of this `LinearOperator`. (deprecated) Warning: THIS FUNCTION IS DEPRECATED. It will be removed in a future version. Instructions for updating: Do not call `graph_parents`.
`is_non_singular`
`is_positive_definite`
`is_self_adjoint`
`is_square`	Return `True/False` depending on if this operator is square.
`range_dimension`	Dimension (in the sense of vector spaces) of the range of this operator. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `M`.
`shape`	`TensorShape` of this `LinearOperator`. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `TensorShape([B1,...,Bb, M, N])`, equivalent to `A.shape`.
`spectrum`
`tensor_rank`	Rank (in the sense of tensors) of matrix corresponding to this operator. If this operator acts like the batch matrix `A` with `A.shape = [B1,...,Bb, M, N]`, then this returns `b + 2`.

Args
`x`	`Tensor` with same `dtype` and shape broadcastable to `self.shape`.
`name`	A name to give this `Op`.

Args
`x`	`LinearOperator` or `Tensor` with compatible shape and same `dtype` as `self`. See class docstring for definition of compatibility.
`adjoint`	Python `bool`. If `True`, left multiply by the adjoint: `A^H x`.
`adjoint_arg`	Python `bool`. If `True`, compute `A x^H` where `x^H` is the hermitian transpose (transposition and complex conjugation).
`name`	A name for this `Op`.

tf.linalg.LinearOperatorCirculant

View aliases

Description in terms of circulant matrices

Description in terms of the frequency spectrum

Operator properties deduced from the spectrum.

Example of a self-adjoint positive definite operator

Example of defining in terms of a real convolution kernel

Example of Hermitian spectrum

Example of forcing real dtype when spectrum is Hermitian

Performance

Matrix property hints

References:

Args

Extension_for_non_symmetric_matrices

Attributes

Methods

add_to_tensor

adjoint

assert_hermitian_spectrum

assert_non_singular

assert_positive_definite

assert_self_adjoint

batch_shape_tensor

block_shape_tensor

cholesky

cond

convolution_kernel

determinant

diag_part

domain_dimension_tensor

eigvals

inverse

log_abs_determinant

matmul

matvec

Example of forcing real `dtype` when spectrum is Hermitian

`add_to_tensor`

`adjoint`

`assert_hermitian_spectrum`

`assert_non_singular`

`assert_positive_definite`

`assert_self_adjoint`

`batch_shape_tensor`

`block_shape_tensor`

`cholesky`

`cond`

`convolution_kernel`

`determinant`

`diag_part`

`domain_dimension_tensor`

`eigvals`

`inverse`

`log_abs_determinant`

`matmul`

`matvec`