tfa.image.sparse_image_warp

Image warping using correspondences between sparse control points.

tfa.image.sparse_image_warp(
    image: tfa.types.TensorLike,
    source_control_point_locations: tfa.types.TensorLike,
    dest_control_point_locations: tfa.types.TensorLike,
    interpolation_order: int = 2,
    regularization_weight: tfa.types.FloatTensorLike = 0.0,
    num_boundary_points: int = 0,
    name: str = 'sparse_image_warp'
) -> tf.Tensor

Apply a non-linear warp to the image, where the warp is specified by the source and destination locations of a (potentially small) number of control points. First, we use a polyharmonic spline (tfa.image.interpolate_spline) to interpolate the displacements between the corresponding control points to a dense flow field. Then, we warp the image using this dense flow field (tfa.image.dense_image_warp).

Let t index our control points. For regularization_weight = 0, we have: warped_image[b, dest_control_point_locations[b, t, 0], dest_control_point_locations[b, t, 1], :] = image[b, source_control_point_locations[b, t, 0], source_control_point_locations[b, t, 1], :].

For regularization_weight > 0, this condition is met approximately, since regularized interpolation trades off smoothness of the interpolant vs. reconstruction of the interpolant at the control points. See tfa.image.interpolate_spline for further documentation of the interpolation_order and regularization_weight arguments.

Args
image	Either a 2-D float Tensor of shape [height, width], a 3-D Tensor of shape [height, width, channels], or a 4-D Tensor of shape [batch_size, height, width, channels]. batch_size is assumed as one when image is a 2-D or 3-D Tensor.
source_control_point_locations	[batch_size, num_control_points, 2] float Tensor.
dest_control_point_locations	[batch_size, num_control_points, 2] float Tensor.
interpolation_order	polynomial order used by the spline interpolation
regularization_weight	weight on smoothness regularizer in interpolation
num_boundary_points	How many zero-flow boundary points to include at each image edge. Usage: num_boundary_points=0: don't add zero-flow points num_boundary_points=1: 4 corners of the image num_boundary_points=2: 4 corners and one in the middle of each edge (8 points total) num_boundary_points=n: 4 corners and n-1 along each edge
name	A name for the operation (optional). Note that image and offsets can be of type tf.half, tf.float32, or tf.float64, and do not necessarily have to be the same type.

Returns
warped_image	a float Tensor with the same shape and dtype as image.
flow_field	[batch_size, height, width, 2] float Tensor containing the dense flow field produced by the interpolation.