tf.keras.layers.Dense

Just your regular densely-connected NN layer.

Inherits From: Layer, Operation

tf.keras.layers.Dense(
    units,
    activation=None,
    use_bias=True,
    kernel_initializer='glorot_uniform',
    bias_initializer='zeros',
    kernel_regularizer=None,
    bias_regularizer=None,
    activity_regularizer=None,
    kernel_constraint=None,
    bias_constraint=None,
    lora_rank=None,
    lora_alpha=None,
    quantization_config=None,
    **kwargs
)

Used in the notebooks

Used in the guide	Used in the tutorials
Advanced automatic differentiation Debug a TensorFlow 2 migrated training pipeline Effective Tensorflow 2 Migration examples: Canned Estimators Migrate early stopping	Overfit and underfit Time series forecasting Load a pandas DataFrame Using DTensors with Keras Intro to Autoencoders

Dense implements the operation: output = activation(dot(input, kernel) + bias) where activation is the element-wise activation function passed as the activation argument, kernel is a weights matrix created by the layer, and bias is a bias vector created by the layer (only applicable if use_bias is True). When this layer is followed by a BatchNormalization layer, it is recommended to set use_bias=False as BatchNormalization has its own bias term.

Note: If the input to the layer has a rank greater than 2, Dense computes the dot product between the inputs and the kernel along the last axis of the inputs and axis 0 of the kernel (using tf.tensordot). For example, if input has dimensions (batch_size, d0, d1), then we create a kernel with shape (d1, units), and the kernel operates along axis 2 of the input, on every sub-tensor of shape (1, 1, d1) (there are batch_size * d0 such sub-tensors). The output in this case will have shape (batch_size, d0, units).

Args
units	Positive integer, dimensionality of the output space.
activation	Activation function to use. If you don't specify anything, no activation is applied (ie. "linear" activation: a(x) = x).
use_bias	Boolean, whether the layer uses a bias vector.
kernel_initializer	Initializer for the kernel weights matrix.
bias_initializer	Initializer for the bias vector.
kernel_regularizer	Regularizer function applied to the kernel weights matrix.
bias_regularizer	Regularizer function applied to the bias vector.
activity_regularizer	Regularizer function applied to the output of the layer (its "activation").
kernel_constraint	Constraint function applied to the kernel weights matrix.
bias_constraint	Constraint function applied to the bias vector.
lora_rank	Optional integer. If set, the layer's forward pass will implement LoRA (Low-Rank Adaptation) with the provided rank. LoRA sets the layer's kernel to non-trainable and replaces it with a delta over the original kernel, obtained via multiplying two lower-rank trainable matrices. This can be useful to reduce the computation cost of fine-tuning large dense layers. You can also enable LoRA on an existing Dense layer by calling layer.enable_lora(rank).
lora_alpha	Optional integer. If set, this parameter scales the low-rank adaptation delta (computed as the product of two lower-rank trainable matrices) during the forward pass. The delta is scaled by lora_alpha / lora_rank, allowing you to fine-tune the strength of the LoRA adjustment independently of lora_rank.

Input shape
N-D tensor with shape: (batch_size, ..., input_dim). The most common situation would be a 2D input with shape (batch_size, input_dim).

Output shape
N-D tensor with shape: (batch_size, ..., units). For instance, for a 2D input with shape (batch_size, input_dim), the output would have shape (batch_size, units).

Attributes
input	Retrieves the input tensor(s) of a symbolic operation. Only returns the tensor(s) corresponding to the first time the operation was called.
kernel
output	Retrieves the output tensor(s) of a layer. Only returns the tensor(s) corresponding to the first time the operation was called.
variable_serialization_spec	Returns a dict mapping quantization modes to variable names in order. This spec is used by save_own_variables and load_own_variables to determine the correct ordering of variables during serialization for each quantization mode. None means no quantization.

Methods

enable_lora

View source

enable_lora(
    rank,
    lora_alpha=None,
    a_initializer='he_uniform',
    b_initializer='zeros'
)

from_config

View source

@classmethod
from_config(
    config
)

Creates an operation from its config.

This method is the reverse of get_config, capable of instantiating the same operation from the config dictionary.

Note: If you override this method, you might receive a serialized dtype config, which is a dict. You can deserialize it as follows:

if "dtype" in config and isinstance(config["dtype"], dict):
    policy = dtype_policies.deserialize(config["dtype"])

Args
config	A Python dictionary, typically the output of get_config.

Returns
An operation instance.