tf.keras.layers.Mosaic

Mosaic implements the Mosaic data augmentation technique.

Inherits From: Layer, Operation

tf.keras.layers.Mosaic(
    offset=(0.25, 0.75),
    seed=None,
    bounding_box_format=None,
    data_format=None,
    **kwargs
)

For every image in the batch, Mosaic picks 3 other images at random from the same batch and arranges the 4 images in a 2x2 grid. A crop the size of a single image is then taken from the grid around a randomly sampled mosaic center, so that each output image contains a piece of each of the 4 images.

Labels are mixed in proportion to the area that each of the 4 images covers in the output. Bounding boxes are moved along with their images and clipped to the output image; boxes that end up entirely outside of it get a label of -1. Segmentation masks are cropped exactly like the images.

A single (unbatched) image is processed as a batch of one image. All 4 images of its grid are then that same image, so the output is the image shifted with wrap-around. To combine different images, call the layer on batches, for example after tf.data.Dataset.batch().

A 3D (unbatched) or 4D (batched) tensor of images, or a dictionary with an "images" key and optional "labels", "bounding_boxes" and "segmentation_masks" keys. Batched images have shape (batch, height, width, channels) in "channels_last" format, or (batch, channels, height, width) in "channels_first" format. Labels have shape (batch, num_classes), or (num_classes,) for a single image, and segmentation masks have the same layout as the images.

Same as the input, except that each image's bounding boxes become 4 times as many, since an output image can contain boxes from 4 images.

offset A float or a tuple of two floats in the range [0, 1]. Controls where the mosaic center can be placed, as a fraction of the image width and height measured from the top-left corner. If a tuple (lower, upper) is passed, the x and y coordinates of the center are each sampled uniformly between lower and upper. If a single float is passed, they are sampled between 0.0 and that value. To always use the same center, pass two identical values, for example (0.5, 0.5). Defaults to (0.25, 0.75).
seed Integer. Used to create a random seed.
bounding_box_format A string specifying the format of the bounding boxes, for example "xyxy". Required if bounding boxes are passed to the layer. Defaults to None.
data_format string, either "channels_last" or "channels_first". The ordering of the dimensions in the inputs. "channels_last" corresponds to inputs with shape (batch, height, width, channels) while "channels_first" corresponds to inputs with shape (batch, channels, height, width). It defaults to the image_data_format value found in your Keras config file at ~/.keras/keras.json. If you never set it, then it will be "channels_last".
**kwargs Base layer keyword arguments, such as name and dtype.

Example:

(images, labels), _ = keras.datasets.cifar10.load_data()
images, labels = images[:8], labels[:8]
labels = keras.ops.cast(keras.ops.one_hot(labels.flatten(), 10), "float32")
mosaic = keras.layers.Mosaic(seed=42)
output = mosaic({"images": images, "labels": labels})

input Retrieves the input tensor(s) of a symbolic operation.

Only returns the tensor(s) corresponding to the first time the operation was called.

output Retrieves the output tensor(s) of a layer.

Only returns the tensor(s) corresponding to the first time the operation was called.

Methods

convert_weight

View source

convert_weight(
    weight
)

Convert the weight if it is from the a different backend.

from_config

View source

@classmethod
from_config(
    config
)

Creates an operation from its config.

This method is the reverse of get_config, capable of instantiating the same operation from the config dictionary.

if "dtype" in config and isinstance(config["dtype"], dict):
    policy = dtype_policies.deserialize(config["dtype"])

Args
config A Python dictionary, typically the output of get_config.

Returns
An operation instance.

get_random_transformation

View source

get_random_transformation(
    data, training=True, seed=None
)

transform_bounding_boxes

View source

transform_bounding_boxes(
    bounding_boxes, transformation, training=True
)

transform_images

View source

transform_images(
    images, transformation, training=True
)

transform_labels

View source

transform_labels(
    labels, transformation, training=True
)

transform_segmentation_masks

View source

transform_segmentation_masks(
    segmentation_masks, transformation, training=True
)

transform_single_bounding_box

View source

transform_single_bounding_box(
    bounding_box, transformation, training=True
)

transform_single_image

View source

transform_single_image(
    image, transformation, training=True
)

transform_single_label

View source

transform_single_label(
    label, transformation, training=True
)

transform_single_segmentation_mask

View source

transform_single_segmentation_mask(
    segmentation_mask, transformation, training=True
)

lora_enabled False