GroupNorm

@frozen
public struct GroupNorm<Scalar> : Layer where Scalar : TensorFlowFloatingPoint

A layer that applies group normalization over a mini-batch of inputs.

Reference: Group Normalization.

  • The offset value, also known as beta.

    Declaration

    public var offset: Tensor<Scalar>
  • The scale value, also known as gamma.

    Declaration

    public var scale: Tensor<Scalar>
  • The number of groups.

    Declaration

    @noDerivative
    public let groupCount: Int
  • The axis where the features lie.

    Declaration

    @noDerivative
    public let axis: Int
  • The variance epsilon value.

    Declaration

    @noDerivative
    public let epsilon: Scalar
  • Creates a group normalization layer.

    Precondition

    The axis cannot be batch axis.

    Precondition

    The offset must have rank 1.

    Precondition

    The number of elements of the offset must be divisible by groups.

    Precondition

    The offset and the scale must have same shape.

    Declaration

    public init(
      offset: Tensor<Scalar>,
      scale: Tensor<Scalar>,
      groupCount: Int,
      axis: Int,
      epsilon: Scalar
    )

    Parameters

    offset

    The initial offset value.

    scale

    The initial scale value.

    groupCount

    The number of groups.

    axis

    The axis where the features lie.

    epsilon

    The variance epsilon value.

  • Creates a group normalization layer.

    Precondition

    The axis cannot be batch axis.

    Precondition

    The feature count must be divisible by groups.

    Declaration

    public init(
      featureCount: Int,
      groupCount: Int,
      axis: Int = -1,
      epsilon: Scalar = 1e-3
    )

    Parameters

    featureCount

    The number of features.

    groupCount

    The number of groups.

    axis

    The axis where the features lie. The default value is -1.

    epsilon

    The small scalar added to variance. The default value is 0.001.

  • Returns the output obtained from applying the layer to the given input.

    Precondition

    The axis cannot be batch axis.

    Precondition

    The numbers of features of the input and the offset must be same.

    Declaration

    @differentiable
    public func forward(_ input: Tensor<Scalar>) -> Tensor<Scalar>

    Parameters

    input

    The input to the layer.

    Return Value

    The output.