tf.keras.initializers.VarianceScaling

Initializer capable of adapting its scale to the shape of weights tensors.

Inherits From: Initializer

tf.keras.initializers.VarianceScaling(
    scale=1.0,
    mode='fan_in',
    distribution='truncated_normal',
    seed=None
)

Also available via the shortcut function tf.keras.initializers.variance_scaling.

With distribution="truncated_normal" or "untruncated_normal", samples are drawn from a truncated/untruncated normal distribution with a mean of zero and a standard deviation (after truncation, if used) stddev = sqrt(scale / n), where n is:

  • number of input units in the weight tensor, if mode="fan_in"
  • number of output units, if mode="fan_out"
  • average of the numbers of input and output units, if mode="fan_avg"

With distribution="uniform", samples are drawn from a uniform distribution within [-limit, limit], where limit = sqrt(3 * scale / n).

Examples:

# Standalone usage:
initializer = tf.keras.initializers.VarianceScaling(
scale=0.1, mode='fan_in', distribution='uniform')
values = initializer(shape=(2, 2))
 
# Usage in a Keras layer:
initializer = tf.keras.initializers.VarianceScaling(
scale=0.1, mode='fan_in', distribution='uniform')
layer = tf.keras.layers.Dense(3, kernel_initializer=initializer)

scale Scaling factor (positive float).
mode One of "fan_in", "fan_out", "fan_avg".
distribution Random distribution to use. One of "truncated_normal", "untruncated_normal" and "uniform".
seed A Python integer. Used to make the behavior of the initializer deterministic. Note that a seeded initializer will not produce the same random values across multiple calls, but multiple initializers will produce the same sequence when constructed with the same seed value.

Methods

from_config

View source

@classmethod
from_config(
    config
)

Instantiates an initializer from a configuration dictionary.

Example:

initializer = RandomUniform(-1, 1)
config = initializer.get_config()
initializer = RandomUniform.from_config(config)

Args
config A Python dictionary, the output of get_config.

Returns
A tf.keras.initializers.Initializer instance.

get_config

View source

get_config()

Returns the configuration of the initializer as a JSON-serializable dict.

Returns
A JSON-serializable Python dict.

__call__

View source

__call__(
    shape, dtype=None, **kwargs
)

Returns a tensor object initialized as specified by the initializer.

Args
shape Shape of the tensor.
dtype Optional dtype of the tensor. Only floating point types are supported. If not specified, tf.keras.backend.floatx() is used, which default to float32 unless you configured it otherwise (via tf.keras.backend.set_floatx(float_dtype))
**kwargs Additional keyword arguments.