Distributed version of Stochastic Dual Coordinate Ascent (SDCA) optimizer for
linear models with L1 + L2 regularization. As global optimization objective is strongly-convex, the optimizer optimizes the dual objective at each step. The optimizer applies each update one example at a time. Examples are sampled uniformly, and the optimizer is learning rate free and enjoys linear convergence rate.
[Proximal Stochastic Dual Coordinate Ascent](http://arxiv.org/pdf/1211.2717v1.pdf).
Shai Shalev-Shwartz, Tong Zhang. 2012
$$Loss Objective = \sum f_{i} (wx_{i}) + (l2 / 2) * |w|^2 + l1 * |w|$$
[Adding vs. Averaging in Distributed Primal-Dual Optimization](http://arxiv.org/abs/1502.03508).
Chenxin Ma, Virginia Smith, Martin Jaggi, Michael I. Jordan,
Peter Richtarik, Martin Takac. 2015
[Stochastic Dual Coordinate Ascent with Adaptive Probabilities](https://arxiv.org/abs/1502.08053).
Dominik Csiba, Zheng Qu, Peter Richtarik. 2015
Nested Classes
| class | SdcaOptimizer.Options | Optional attributes for SdcaOptimizer
|
|
Constants
| String | OP_NAME | The name of this op, as known by TensorFlow core engine |
Public Methods
| static SdcaOptimizer.Options |
adaptive(Boolean adaptive)
|
| static SdcaOptimizer |
create(Scope scope, Iterable<Operand<TInt64>> sparseExampleIndices, Iterable<Operand<TInt64>> sparseFeatureIndices, Iterable<Operand<TFloat32>> sparseFeatureValues, Iterable<Operand<TFloat32>> denseFeatures, Operand<TFloat32> exampleWeights, Operand<TFloat32> exampleLabels, Iterable<Operand<TInt64>> sparseIndices, Iterable<Operand<TFloat32>> sparseWeights, Iterable<Operand<TFloat32>> denseWeights, Operand<TFloat32> exampleStateData, String lossType, Float l1, Float l2, Long numLossPartitions, Long numInnerIterations, Options... options)
Factory method to create a class wrapping a new SdcaOptimizer operation.
|
| List<Output<TFloat32>> |
outDeltaDenseWeights()
a list of vectors where the values are the delta
weights associated with a dense feature group.
|
| List<Output<TFloat32>> |
outDeltaSparseWeights()
a list of vectors where each value is the delta
weights associated with a sparse feature group.
|
| Output<TFloat32> |
outExampleStateData()
a list of vectors containing the updated example state
data.
|
Inherited Methods
Constants
public static final String OP_NAME
The name of this op, as known by TensorFlow core engine
Public Methods
public static SdcaOptimizer.Options adaptive (Boolean adaptive)
Parameters
| adaptive | Whether to use Adaptive SDCA for the inner loop. |
|---|
public static SdcaOptimizer create (Scope scope, Iterable<Operand<TInt64>> sparseExampleIndices, Iterable<Operand<TInt64>> sparseFeatureIndices, Iterable<Operand<TFloat32>> sparseFeatureValues, Iterable<Operand<TFloat32>> denseFeatures, Operand<TFloat32> exampleWeights, Operand<TFloat32> exampleLabels, Iterable<Operand<TInt64>> sparseIndices, Iterable<Operand<TFloat32>> sparseWeights, Iterable<Operand<TFloat32>> denseWeights, Operand<TFloat32> exampleStateData, String lossType, Float l1, Float l2, Long numLossPartitions, Long numInnerIterations, Options... options)
Factory method to create a class wrapping a new SdcaOptimizer operation.
Parameters
| scope | current scope |
|---|---|
| sparseExampleIndices | a list of vectors which contain example indices. |
| sparseFeatureIndices | a list of vectors which contain feature indices. |
| sparseFeatureValues | a list of vectors which contains feature value associated with each feature group. |
| denseFeatures | a list of matrices which contains the dense feature values. |
| exampleWeights | a vector which contains the weight associated with each example. |
| exampleLabels | a vector which contains the label/target associated with each example. |
| sparseIndices | a list of vectors where each value is the indices which has corresponding weights in sparse_weights. This field maybe omitted for the dense approach. |
| sparseWeights | a list of vectors where each value is the weight associated with a sparse feature group. |
| denseWeights | a list of vectors where the values are the weights associated with a dense feature group. |
| exampleStateData | a list of vectors containing the example state data. |
| lossType | Type of the primal loss. Currently SdcaSolver supports logistic, squared and hinge losses. |
| l1 | Symmetric l1 regularization strength. |
| l2 | Symmetric l2 regularization strength. |
| numLossPartitions | Number of partitions of the global loss function. |
| numInnerIterations | Number of iterations per mini-batch. |
| options | carries optional attributes values |
Returns
- a new instance of SdcaOptimizer
public List<Output<TFloat32>> outDeltaDenseWeights ()
a list of vectors where the values are the delta weights associated with a dense feature group.
public List<Output<TFloat32>> outDeltaSparseWeights ()
a list of vectors where each value is the delta weights associated with a sparse feature group.