![]() |
![]() |
LinearOperator
acting like a circulant matrix.
tf.linalg.LinearOperatorCirculant(
spectrum, input_output_dtype=tf.dtypes.complex64, is_non_singular=None,
is_self_adjoint=None, is_positive_definite=None, is_square=True,
name='LinearOperatorCirculant'
)
This operator acts like a circulant matrix A
with
shape [B1,...,Bb, N, N]
for some b >= 0
. The first b
indices index a
batch member. For every batch index (i1,...,ib)
, A[i1,...,ib, : :]
is
an N x N
matrix. This matrix A
is not materialized, but for
purposes of broadcasting this shape will be relevant.
Description in terms of circulant matrices
Circulant means the entries of A
are generated by a single vector, the
convolution kernel h
: A_{mn} := h_{m-n mod N}
. With h = [w, x, y, z]
,
A = |w z y x|
|x w z y|
|y x w z|
|z y x w|
This means that the result of matrix multiplication v = Au
has Lth
column
given circular convolution between h
with the Lth
column of u
.
Description in terms of the frequency spectrum
There is an equivalent description in terms of the [batch] spectrum H
and
Fourier transforms. Here we consider A.shape = [N, N]
and ignore batch
dimensions. Define the discrete Fourier transform (DFT) and its inverse by
DFT[ h[n] ] = H[k] := sum_{n = 0}^{N - 1} h_n e^{-i 2pi k n / N}
IDFT[ H[k] ] = h[n] = N^{-1} sum_{k = 0}^{N - 1} H_k e^{i 2pi k n / N}
From these definitions, we see that
H[0] = sum_{n = 0}^{N - 1} h_n
H[1] = "the first positive frequency"
H[N - 1] = "the first negative frequency"
Loosely speaking, with *
element-wise multiplication, matrix multiplication
is equal to the action of a Fourier multiplier: A u = IDFT[ H * DFT[u] ]
.
Precisely speaking, given [N, R]
matrix u
, let DFT[u]
be the [N, R]
matrix with rth
column equal to the DFT of the rth
column of u
.
Define the IDFT
similarly.
Matrix multiplication may be expressed columnwise:
Operator properties deduced from the spectrum.
Letting U
be the kth
Euclidean basis vector, and U = IDFT[u]
.
The above formulas show thatA U = H_k * U
. We conclude that the elements
of H
are the eigenvalues of this operator. Therefore
- This operator is positive definite if and only if
Real{H} > 0
.
A general property of Fourier transforms is the correspondence between Hermitian functions and real valued transforms.
Suppose H.shape = [B1,...,Bb, N]
. We say that H
is a Hermitian spectrum
if, with %
meaning modulus division,
- This operator corresponds to a real matrix if and only if
H
is Hermitian. - This operator is self-adjoint if and only if
H
is real.
See e.g. "Discrete-Time Signal Processing", Oppenheim and Schafer.
Example of a self-adjoint positive definite operator
# spectrum is real ==> operator is self-adjoint
# spectrum is positive ==> operator is positive definite
spectrum = [6., 4, 2]
operator = LinearOperatorCirculant(spectrum)
# IFFT[spectrum]
operator.convolution_kernel()
==> [4 + 0j, 1 + 0.58j, 1 - 0.58j]
operator.to_dense()
==> [[4 + 0.0j, 1 - 0.6j, 1 + 0.6j],
[1 + 0.6j, 4 + 0.0j, 1 - 0.6j],
[1 - 0.6j, 1 + 0.6j, 4 + 0.0j]]
Example of defining in terms of a real convolution kernel
# convolution_kernel is real ==> spectrum is Hermitian.
convolution_kernel = [1., 2., 1.]]
spectrum = tf.signal.fft(tf.cast(convolution_kernel, tf.complex64))
# spectrum is Hermitian ==> operator is real.
# spectrum is shape [3] ==> operator is shape [3, 3]
# We force the input/output type to be real, which allows this to operate
# like a real matrix.
operator = LinearOperatorCirculant(spectrum, input_output_dtype=tf.float32)
operator.to_dense()
==> [[ 1, 1, 2],
[ 2, 1, 1],
[ 1, 2, 1]]
Example of Hermitian spectrum
# spectrum is shape [3] ==> operator is shape [3, 3]
# spectrum is Hermitian ==> operator is real.
spectrum = [1, 1j, -1j]
operator = LinearOperatorCirculant(spectrum)
operator.to_dense()
==> [[ 0.33 + 0j, 0.91 + 0j, -0.24 + 0j],
[-0.24 + 0j, 0.33 + 0j, 0.91 + 0j],
[ 0.91 + 0j, -0.24 + 0j, 0.33 + 0j]
Example of forcing real dtype
when spectrum is Hermitian
# spectrum is shape [4] ==> operator is shape [4, 4]
# spectrum is real ==> operator is self-adjoint
# spectrum is Hermitian ==> operator is real
# spectrum has positive real part ==> operator is positive-definite.
spectrum = [6., 4, 2, 4]
# Force the input dtype to be float32.
# Cast the output to float32. This is fine because the operator will be
# real due to Hermitian spectrum.
operator = LinearOperatorCirculant(spectrum, input_output_dtype=tf.float32)
operator.shape
==> [4, 4]
operator.to_dense()
==> [[4, 1, 0, 1],
[1, 4, 1, 0],
[0, 1, 4, 1],
[1, 0, 1, 4]]
# convolution_kernel = tf.signal.ifft(spectrum)
operator.convolution_kernel()
==> [4, 1, 0, 1]
Performance
Suppose operator
is a LinearOperatorCirculant
of shape [N, N]
,
and x.shape = [N, R]
. Then
operator.matmul(x)
isO(R*N*Log[N])
operator.solve(x)
isO(R*N*Log[N])
operator.determinant()
involves a sizeN
reduce_prod
.
If instead operator
and x
have shape [B1,...,Bb, N, N]
and
[B1,...,Bb, N, R]
, every operation increases in complexity by B1*...*Bb
.
Matrix property hints
This LinearOperator
is initialized with boolean flags of the form is_X
,
for X = non_singular, self_adjoint, positive_definite, square
.
These have the following meaning:
- If
is_X == True
, callers should expect the operator to have the propertyX
. This is a promise that should be fulfilled, but is not a runtime assert. For example, finite floating point precision may result in these promises being violated. - If
is_X == False
, callers should expect the operator to not haveX
. - If
is_X == None
(the default), callers should have no expectation either way.
References:
Toeplitz and Circulant Matrices - A Review: Gray, 2006 (pdf)
Args | |
---|---|
spectrum
|
Shape [B1,...,Bb, N] Tensor . Allowed dtypes: float16 ,
float32 , float64 , complex64 , complex128 . Type can be different
than input_output_dtype
|
input_output_dtype
|
dtype for input/output.
|
is_non_singular
|
Expect that this operator is non-singular. |
is_self_adjoint
|
Expect that this operator is equal to its hermitian
transpose. If spectrum is real, this will always be true.
|
is_positive_definite
|
Expect that this operator is positive definite,
meaning the quadratic form x^H A x has positive real part for all
nonzero x . Note that we do not require the operator to be
self-adjoint to be positive-definite. See:
https://en.wikipedia.org/wiki/Positive-definite_matrix\
Extension_for_non_symmetric_matrices |
is_square
|
Expect that this operator acts like square [batch] matrices. |
name
|
A name to prepend to all ops created by this class. |
Attributes | |
---|---|
H
|
Returns the adjoint of the current LinearOperator .
Given |
batch_shape
|
TensorShape of batch dimensions of this LinearOperator .
If this operator acts like the batch matrix |
block_depth
|
Depth of recursively defined circulant blocks defining this Operator .
With
|
block_shape
|
|
domain_dimension
|
Dimension (in the sense of vector spaces) of the domain of this operator.
If this operator acts like the batch matrix |
dtype
|
The DType of Tensor s handled by this LinearOperator .
|
graph_parents
|
List of graph dependencies of this LinearOperator . (deprecated)
|
is_non_singular
|
|
is_positive_definite
|
|
is_self_adjoint
|
|
is_square
|
Return True/False depending on if this operator is square.
|
range_dimension
|
Dimension (in the sense of vector spaces) of the range of this operator.
If this operator acts like the batch matrix |
shape
|
TensorShape of this LinearOperator .
If this operator acts like the batch matrix |
spectrum
|
|
tensor_rank
|
Rank (in the sense of tensors) of matrix corresponding to this operator.
If this operator acts like the batch matrix |
Methods
add_to_tensor
add_to_tensor(
x, name='add_to_tensor'
)
Add matrix represented by this operator to x
. Equivalent to A + x
.
Args | |
---|---|
x
|
Tensor with same dtype and shape broadcastable to self.shape .
|
name
|
A name to give this Op .
|
Returns | |
---|---|
A Tensor with broadcast shape and same dtype as self .
|
adjoint
adjoint(
name='adjoint'
)
Returns the adjoint of the current LinearOperator
.
Given A
representing this LinearOperator
, return A*
.
Note that calling self.adjoint()
and self.H
are equivalent.
Args | |
---|---|
name
|
A name for this Op .
|
Returns | |
---|---|
LinearOperator which represents the adjoint of this LinearOperator .
|
assert_hermitian_spectrum
assert_hermitian_spectrum(
name='assert_hermitian_spectrum'
)
Returns an Op
that asserts this operator has Hermitian spectrum.
This operator corresponds to a real-valued matrix if and only if its spectrum is Hermitian.
Args | |
---|---|
name
|
A name to give this Op .
|
Returns | |
---|---|
An Op that asserts this operator has Hermitian spectrum.
|
assert_non_singular
assert_non_singular(
name='assert_non_singular'
)
Returns an Op
that asserts this operator is non singular.
This operator is considered non-singul