tf.keras.layers.Layer

TensorFlow 2 version

View source on GitHub

Base layer class.

Inherits From: Module

View aliases

Compat aliases for migration

See Migration guide for more details.

tf.compat.v1.keras.layers.Layer, `tf.compat.v2.keras.layers.Layer`

tf.keras.layers.Layer(
    trainable=True, name=None, dtype=None, dynamic=False, **kwargs
)

This is the class from which all layers inherit.

A layer is a class implementing common neural networks operations, such as convolution, batch norm, etc. These operations require managing weights, losses, updates, and inter-layer connectivity.

Users will just instantiate a layer and then treat it as a callable.

We recommend that descendants of Layer implement the following methods:

__init__(): Save configuration in member variables
build(): Called once from __call__, when we know the shapes of inputs and dtype. Should have the calls to add_weight(), and then call the super's build() (which sets self.built = True, which is nice in case the user wants to call build() manually before the first __call__).
call(): Called in __call__ after making sure build() has been called once. Should actually perform the logic of applying the layer to the input tensors (which should be passed in as the first argument).

Arguments
`trainable`	Boolean, whether the layer's variables should be trainable.
`name`	String name of the layer.
`dtype`	The dtype of the layer's computations and weights (default of `None` means use `tf.keras.backend.floatx` in TensorFlow 2, or the type of the first input in TensorFlow 1).
`dynamic`	Set this to `True` if your layer should only be run eagerly, and should not be used to generate a static computation graph. This would be the case for a Tree-RNN or a recursive network, for example, or generally for any layer that manipulates tensors using Python control flow. If `False`, we assume that the layer can safely be used to generate a static computation graph.

Read-only properties: name: The name of the layer (string). dtype: The dtype of the layer's computations and weights. If mixed precision is used with a tf.keras.mixed_precision.experimental.Policy, this is instead just the dtype of the layer's weights, as the computations are done in a different dtype. updates: List of update ops of this layer. losses: List of losses added by this layer. trainable_weights: List of variables to be included in backprop. non_trainable_weights: List of variables that should not be included in backprop. weights: The concatenation of the lists trainable_weights and non_trainable_weights (in this order).

Mutable properties:

trainable: Whether the layer should be trained (boolean).
input_spec: Optional (list of) InputSpec object(s) specifying the constraints on inputs that can be accepted by the layer.

Dtypes and casting

Each layer has a dtype, which is typically the dtype of the layer's computations and variables. A layer's dtype can be queried via the Layer.dtype property. The dtype is specified with the dtype constructor argument. In TensorFlow 2, the dtype defaults to tf.keras.backend.floatx() if no dtype is passed. floatx() itself defaults to "float32". Additionally, layers will cast their inputs to the layer's dtype in TensorFlow 2. For example:

x = tf.ones((4, 4, 4, 4), dtype='float64')
layer = tf.keras.layers.Conv2D(filters=4, kernel_size=2)
print(layer.dtype)  # float32

# `layer` casts it's inputs to layer.dtype, which is float32, and does
# computations in float32.
y = layer(x)

Currently, only tensors in the first argument to the layer's call method are casted. For example:

class MyLayer(tf.keras.layers.Layer):
  # Bug! `b` will not be casted.
  def call(self, a, b):
    return a + 1., b + 1.

a = tf.constant(1., dtype="float32")
b = tf.constant(1., dtype="float32")

layer = MyLayer(dtype="float64")
x, y = layer(a, b)
print(x.dtype)  # float64
print(y.dtype)  # float32. Not casted since `b` was not passed to first input

It is recommended to accept tensors only in the first argument. This way, all tensors are casted to the layer's dtype. MyLayer should therefore be written as:

class MyLayer(tf.keras.layers.Layer):
  # Now, all tensor inputs will be casted.
  def call(self, inputs):
    a, b = inputs
    return a + 1., b + 1.

a = tf.constant(1., dtype="float32")
b = tf.constant(1., dtype="float32")

layer = MyLayer(dtype="float64")
x, y = layer((a, b))
print(x.dtype)  # float64
print(y.dtype)  # float64.

In a future minor release, tensors in other arguments may be casted as well.

Currently, other arguments are not automatically casted for technical reasons, but this may change in a future minor release.

A layer subclass can prevent its inputs from being autocasted by passing autocast=False to the layer constructor. For example:

class MyLayer(tf.keras.layers.Layer):

  def __init__(self, **kwargs):
    kwargs['autocast']=False
    super(MyLayer, self).__init__(**kwargs)

  def call(self, inp):
    return inp

x = tf.ones((4, 4, 4, 4), dtype='float64')
layer = MyLayer()
print(layer.dtype)  # float32.
y = layer(x)  # MyLayer will not cast inputs to it's dtype of float32
print(y.dtype)  # float64

Running models in float64 in TensorFlow 2

If you want to run a Model in float64, you can set floatx to be float64 by calling tf.keras.backend.set_floatx('float64'). This will cause all layers to default to float64 instead of float32:

tf.keras.backend.set_floatx('float64')
layer1 = tf.keras.layers.Dense(4)
layer2 = tf.keras.layers.Dense(4)

x = tf.ones((4, 4))
y = layer2(layer1(x))  # Both layers run in float64

Alternatively, you can pass dtype='float64' to each individual layer. Note that if you have any layers which contain other layers as members, you must ensure each sublayer gets dtype='float64' passed to it's constructor as well:

layer1 = tf.keras.layers.Dense(4, dtype='float64')
layer2 = tf.keras.layers.Dense(4, dtype='float64')

x = tf.ones((4, 4))
y = layer2(layer1(x))  # Both layers run in float64

class NestedLayer(tf.keras.layers.Layer):
  def __init__(self, **kwargs):
    super(NestedLayer, self).__init__(**kwargs)
    self.dense = tf.keras.layers.Dense(4, dtype=kwargs.get('dtype'))

  def call(self, inp):
    return self.dense(inp)

layer3 = NestedLayer(dtype='float64')
z = layer3(x)  # layer3's dense layer runs in float64, since NestedLayer
               # correcty passed it's dtype to it's dense layer

Attributes
`activity_regularizer`	Optional regularizer function for the output of this layer.
`dtype`
`dynamic`
`input`	Retrieves the input tensor(s) of a layer. Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer.
`input_mask`	Retrieves the input mask tensor(s) of a layer. Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.
`input_shape`	Retrieves the input shape(s) of a layer. Only applicable if the layer has exactly one input, i.e. if it is connected to one incoming layer, or if all inputs have the same shape.
`input_spec`
`losses`	Losses which are associated with this `Layer`. Variable regularization tensors are created when this property is accessed, so it is eager safe: accessing `losses` under a `tf.GradientTape` will propagate gradients back to the corresponding variables.
`metrics`
`name`	Returns the name of this module as passed or determined in the ctor. Note: This is not the same as the `self.name_scope.name` which includes parent module names.
`non_trainable_variables`
`non_trainable_weights`
`output`	Retrieves the output tensor(s) of a layer. Only applicable if the layer has exactly one output, i.e. if it is connected to one incoming layer.
`output_mask`	Retrieves the output mask tensor(s) of a layer. Only applicable if the layer has exactly one inbound node, i.e. if it is connected to one incoming layer.
`output_shape`	Retrieves the output shape(s) of a layer. Only applicable if the layer has one output, or if all outputs have the same shape.
`trainable`
`trainable_variables`	Sequence of variables owned by this module and it's submodules. Note: this method uses reflection to find variables on the current instance and submodules. For performance reasons you may wish to cache the result of calling this method if you don't expect the return value to change.
`trainable_weights`
`updates`
`variables`	Returns the list of all layer variables/weights. Alias of `self.weights`.
`weights`	Returns the list of all layer variables/weights.

Args
`value`	Metric tensor.
`aggregation`	Sample-wise metric reduction function. If `aggregation=None`, it indicates that the metric tensor provided has been aggregated already. eg, `bin_acc = BinaryAccuracy(name='acc')` followed by `model.add_metric(bin_acc(y_true, y_pred))`. If aggregation='mean', the given metric tensor will be sample-wise reduced using `mean` function. eg, `model.add_metric(tf.reduce_sum(outputs), name='output_mean', aggregation='mean')`.
`name`	String metric name.

Arguments
`updates`	Update op, or list/tuple of update ops, or zero-arg callable that returns an update op. A zero-arg callable should be passed in order to disable running the updates by setting `trainable=False` on this Layer, when executing in Eager mode.
`inputs`	Deprecated, will be automatically inferred.

Arguments
`name`	Variable name.
`shape`	Variable shape. Defaults to scalar if unspecified.
`dtype`	The type of the variable. Defaults to `self.dtype` or `float32`.
`initializer`	Initializer instance (callable).
`regularizer`	Regularizer instance (callable).
`trainable`	Boolean, whether the variable should be part of the layer's "trainable_variables" (e.g. variables, biases) or "non_trainable_variables" (e.g. BatchNorm mean and variance). Note that `trainable` cannot be `True` if `synchronization` is set to `ON_READ`.
`constraint`	Constraint instance (callable).
`partitioner`	Partitioner to be passed to the `Trackable` API.
`use_resource`	Whether to use `ResourceVariable`.
`synchronization`	Indicates when a distributed a variable will be aggregated. Accepted values are constants defined in the class `tf.VariableSynchronization`. By default the synchronization is set to `AUTO` and the current `DistributionStrategy` chooses when to synchronize. If `synchronization` is set to `ON_READ`, `trainable` must not be set to `True`.
`aggregation`	Indicates how a distributed variable will be aggregated. Accepted values are constants defined in the class `tf.VariableAggregation`.
`**kwargs`	Additional keyword arguments. Accepted values are `getter` and `collections`.

Raises
`RuntimeError`	If called with partitioned variable regularization and eager execution is enabled.
`ValueError`	When giving unsupported dtype and no initializer or when trainable has been set to True with synchronization set as `ON_READ`.

Arguments
`inputs`	Input tensor, or list/tuple of input tensors.
`**kwargs`	Additional keyword arguments.

Arguments
`inputs`	Tensor or list of tensors.
`mask`	Tensor or list of tensors.

Arguments
`inputs`	input tensor(s).
`*args`	additional positional arguments to be passed to `self.call`.
`**kwargs`	additional keyword arguments to be passed to `self.call`.

tf.keras.layers.Layer

View aliases

Arguments

Mutable properties:

Dtypes and casting

Running models in float64 in TensorFlow 2

Attributes

Methods

add_loss

Example:

Example:

Example:

add_metric

add_update

add_weight

build

call

compute_mask

compute_output_shape

compute_output_signature

count_params

from_config

get_config

get_input_at

get_input_mask_at

get_input_shape_at

get_losses_for

get_output_at

get_output_mask_at

get_output_shape_at

get_updates_for

get_weights

set_weights

__call__

Note:

`add_loss`

`add_metric`

`add_update`

`add_weight`

`build`

`call`

`compute_mask`

`compute_output_shape`

`compute_output_signature`

`count_params`

`from_config`

`get_config`

`get_input_at`

`get_input_mask_at`

`get_input_shape_at`

`get_losses_for`

`get_output_at`

`get_output_mask_at`

`get_output_shape_at`

`get_updates_for`

`get_weights`

`set_weights`

`call`