Differentiable

public protocol Differentiable

A type that mathematically represents a differentiable manifold whose tangent spaces are finite-dimensional.

• TangentVector

  A type representing a differentiable value’s derivatives.

  Mathematically, this is equivalent to the tangent bundle of the differentiable manifold represented by the differentiable type.

  Declaration

  associatedtype TangentVector: Differentiable & AdditiveArithmetic
    where TangentVector.TangentVector == TangentVector

• move(along:)

  Moves self along the given direction. In Riemannian geometry, this is equivalent to exponential map, which moves self on the geodesic surface along the given tangent vector.

  Declaration

  mutating mutating func move(along direction: TangentVector)

• zeroTangentVectorInitializer

  A closure that produces a zero tangent vector, capturing minimal necessary information from self.

  move(along: zeroTangentVectorInitializer()) should not modify self.

  In some cases, the zero tangent vector of self is equal to TangentVector.zero. In other cases, the zero tangent vector depends on information in self, such as shape for an n-dimensional array type. For differentiable programming, it is more memory-efficient to define a custom zeroTangentVectorInitializer property which returns a closure that captures and uses only the necessary information to create a zero tangent vector. For example:

  struct Vector {
      var scalars: [Float]
      var count: Int { scalars.count }
      init(scalars: [Float]) { ... }
      init(repeating repeatedElement: Float, count: Int) { ... }
  }
  
  extension Vector: AdditiveArithmetic { ... }
  
  extension Vector: Differentiable {
      typealias TangentVector = Vector
  
      @noDerivative
      var zeroTangentVectorInitializer: () -> TangentVector {
          let count = self.count
          return { TangentVector(repeating: 0, count: count) }
      }
  }
  

  Declaration

  var zeroTangentVectorInitializer: () -> TangentVector { get }

• zeroTangentVector

  Extension method
  A tangent vector initialized using zeroTangentVectorInitializer. move(along: zeroTangentVector) should not modify self.

  Declaration

  var zeroTangentVector: TangentVector { get }

• withRecomputationInPullbacks(_:)

  Extension method
  Declaration

  @differentiable(wrt: self)
  func withRecomputationInPullbacks<Result : Differentiable>(
    _ body: @escaping @differentiable (Self) -> Result
  ) -> Result

• withDerivative(_:)

  Extension method
  Applies the given closure to the derivative of self.

  Returns self like an identity function. When the return value is used in a context where it is differentiated with respect to, applies the given closure to the derivative of the return value.

  Declaration

  @differentiable(wrt: self)
  func withDerivative(_ body: @escaping (inout TangentVector) -> Void) -> Self

• sequenced(through:_:)

  Extension method
  Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

  Declaration

  @differentiable
  public func sequenced<L1: Layer, L2: Layer>(through l1: L1, _ l2: L2) -> L2.Output
  where L1.Input == Self, L1.Output == L2.Input

  Parameters

  l1	The first layer.
  l2	The second layer.

  Return Value

  The final layer’s output after sequential application.

• sequenced(through:_:_:)

  Extension method
  Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

  Declaration

  @differentiable
  public func sequenced<L1: Layer, L2: Layer, L3: Layer>(through l1: L1, _ l2: L2, _ l3: L3)
    -> L3.Output
  where L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input

  Parameters

  l1	The first layer.
  l2	The second layer.
  l3	The third layer.

  Return Value

  The final layer’s output after sequential application.

• sequenced(through:_:_:_:)

  Extension method
  Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

  Declaration

  @differentiable
  public func sequenced<L1: Layer, L2: Layer, L3: Layer, L4: Layer>(
    through l1: L1, _ l2: L2, _ l3: L3, _ l4: L4
  ) -> L4.Output
  where
    L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input,
    L3.Output == L4.Input

  Parameters

  l1	The first layer.
  l2	The second layer.
  l3	The third layer.
  l4	The fourth layer.

  Return Value

  The final layer’s output after sequential application.

• sequenced(through:_:_:_:_:)

  Extension method
  Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

  Declaration

  @differentiable
  public func sequenced<L1: Layer, L2: Layer, L3: Layer, L4: Layer, L5: Layer>(
    through l1: L1, _ l2: L2, _ l3: L3, _ l4: L4, _ l5: L5
  ) -> L5.Output
  where
    L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input, L3.Output == L4.Input,
    L4.Output == L5.Input

  Parameters

  l1	The first layer.
  l2	The second layer.
  l3	The third layer.
  l4	The third layer.
  l5	The fifth layer.

  Return Value

  The final layer’s output after sequential application.

• sequenced(through:_:_:_:_:_:)

  Extension method
  Returns the output computed by applying a sequence of layers to the previous layer’s output, except that the first layer’s input is self.

  Declaration

  @differentiable
  public func sequenced<L1: Layer, L2: Layer, L3: Layer, L4: Layer, L5: Layer, L6: Layer>(
    through l1: L1, _ l2: L2, _ l3: L3, _ l4: L4, _ l5: L5, _ l6: L6
  ) -> L6.Output
  where
    L1.Input == Self, L1.Output == L2.Input, L2.Output == L3.Input, L3.Output == L4.Input,
    L4.Output == L5.Input, L5.Output == L6.Input

  Parameters

  l1	The first layer.
  l2	The second layer.
  l3	The third layer.
  l4	The third layer.
  l5	The fifth layer.
  l6	The sixth layer.

  Return Value

  The final layer’s output after sequential application.