nsl.lib.normalize

Normalizes the values in tensor with respect to a specified vector norm.

nsl.lib.normalize(
    tensor, norm_type, epsilon=1e-06
)

This op assumes that the first axis of tensor is the batch dimension, and calculates the norm over all other axes. For example, if tensor is tf.constant(1.0, shape=[2, 3, 4]), its L2 norm (calculated along all the dimensions other than the first dimension) will be [[sqrt(12)], [sqrt(12)]]. Hence, this tensor will be normalized by dividing by [[sqrt(12)], [sqrt(12)]].

Note that tf.norm is not used here since it only allows the norm to be calculated over one axis, not multiple axes.

tensor a tensor to be normalized. Can have any shape with the first axis being the batch dimension that will not be normalized across.
norm_type one of nsl.configs.NormType, the type of vector norm.
epsilon a lower bound value for the norm to avoid division by 0.

A normalized tensor with the same shape and type as tensor.