Help protect the Great Barrier Reef with TensorFlow on Kaggle Join Challenge

tf.contrib.factorization.WALSMatrixFactorization

View source on GitHub

An Estimator for Weighted Matrix Factorization, using the WALS method.

Inherits From: Estimator

WALS (Weighted Alternating Least Squares) is an algorithm for weighted matrix factorization. It computes a low-rank approximation of a given sparse (n x m) matrix A, by a product of two matrices, U * V^T, where U is a (n x k) matrix and V is a (m x k) matrix. Here k is the rank of the approximation, also called the embedding dimension. We refer to U as the row factors, and V as the column factors. See tensorflow/contrib/factorization/g3doc/wals.md for the precise problem formulation.

The training proceeds in sweeps: during a row_sweep, we fix V and solve for U. During a column sweep, we fix U and solve for V. Each one of these problems is an unconstrained quadratic minimization problem and can be solved exactly (it can also be solved in mini-batches, since the solution decouples across rows of each matrix). The alternating between sweeps is achieved by using a hook during training, which is responsible for keeping track of the sweeps and running preparation ops at the beginning of each sweep. It also updates the global_step variable, which keeps track of the number of batches processed since the beginning of training. The current implementation assumes that the training is run on a single machine, and will fail if config.num_worker_replicas is not equal to one. Training is done by calling self.fit(input_fn=input_fn), where input_fn provides two tensors: one for rows of the input matrix, and one for rows of the transposed input matrix (i.e. columns of the original matrix). Note that during a row sweep, only row batches are processed (ignoring column batches) and vice-versa. Also note that every row (respectively every column) of the input matrix must be processed at least once for the sweep to be considered complete. In particular, training will not make progress if some rows are not generated by the input_fn.

For prediction, given a new set of input rows A', we compute a corresponding set of row factors U', such that U' * V^T is a good approximation of A'. We call this operation a row projection. A similar operation is defined for columns. Projection is done by calling self.get_projections(input_fn=input_fn), where input_fn satisfies the constraints given below.

The input functions must satisfy the following constraints: Calling input_fn must return a tuple (features, labels) where labels is None, and features is a dict containing the following keys:

TRAIN:

  • WALSMatrixFactorization.INPUT_ROWS: float32 SparseTensor (matrix). Rows of the input matrix to process (or to project).
  • WALSMatrixFactorization.INPUT_COLS: float32 SparseTensor (matrix). Columns of the input matrix to process (or to project), transposed.

INFER:

  • WALSMatrixFactorization.INPUT_ROWS: float32 SparseTensor (matrix). Rows to project.
  • WALSMatrixFactorization.INPUT_COLS: float32 SparseTensor (matrix). Columns to project.
  • WALSMatrixFactorization.PROJECT_ROW: Boolean Tensor. Whether to project the rows or columns.
  • WALSMatrixFactorization.PROJECTION_WEIGHTS (Optional): float32 Tensor (vector). The weights to use in the projection.

EVAL:

  • WALSMatrixFactorization.INPUT_ROWS: float32 SparseTensor (matrix). Rows to project.
  • WALSMatrixFactorization.INPUT_COLS: float32 SparseTensor (matrix). Columns to project.
  • WALSMatrixFactorization.PROJECT_ROW: Boolean Tensor. Whether to project the rows or columns.

num_rows Total number of rows for input matrix.
num_cols Total number of cols for input matrix.
embedding_dimension Dimension to use for the factors.
unobserved_weight Weight of the unobserved entries of matrix.
regularization_coeff Weight of the L2 regularization term. Defaults to None, in which case the problem is not regularized.
row_init Initializer for row factor. Must be either:

  • A tensor: The row factor matrix is initialized to this tensor,
  • A numpy constant,
  • "random": The rows are initialized using a normal distribution.
col_init Initializer for column factor. See row_init.
num_row_shards Number of shards to use for the row factors.
num_col_shards Number of shards to use for the column factors.
row_weights Must be in one of the following three formats:
  • None: In this case, the weight of every entry is the unobserved_weight and the problem simplifies to ALS. Note that, in this case, col_weights must also be set to "None".
  • List of lists of non-negative scalars, of the form \([[w_0, w_1, ...], [w_k, ... ], [...]]\), where the number of inner lists equal to the number of row factor shards and the elements in each inner list are the weights for the rows of that shard. In this case, \(w_ij = unonbserved_weight + row_weights[i] * col_weights[j]\).
  • A non-negative scalar: This value is used for all row weights. Note that it is allowed to have row_weights as a list and col_weights as a scalar, or vice-versa.
  • col_weights See row_weights.
    use_factors_weights_cache_for_training Boolean, whether the factors and weights will be cached on the workers before the updates start, during training. Defaults to True. Note that caching is disabled during prediction.
    use_gramian_cache_for_training Boolean, whether the Gramians will be cached on the workers before the updates start, during training. Defaults to True. Note that caching is disabled during prediction.
    max_sweeps integer, optional. Specifies the number of sweeps for which to train the model, where a sweep is defined as a full update of all the row factors (resp. column factors). If steps or max_steps is also specified in model.fit(), training stops when either of the steps condition or sweeps condition is met.
    model_dir The directory to save the model results and log files.
    config A Configuration object. See Estimator.

    ValueError If config.num_worker_replicas is strictly greater than one. The current implementation only supports running on a single worker.

    config

    model_dir Returns a path in which the eval process will look for checkpoints.
    model_fn Returns the model_fn which is bound to self.params.

    Methods

    evaluate

    View source

    See Evaluable. (deprecated arguments)

    Raises
    ValueError If at least one of x or y is provided, and at least one of input_fn or feed_fn is provided. Or if metrics is not None or dict.

    export

    View source

    Exports inference graph into given dir. (deprecated)

    Args
    export_dir A string containing a directory to write the exported graph and checkpoints.
    input_fn If use_deprecated_input_fn is true, then a function that given Tensor of Example strings, parses it into features that are then passed to the model. Otherwise, a function that takes no argument and returns a tuple of (features, labels), where features is a dict of string key to Tensor and labels is a Tensor that's currently not used (and so can be None).
    input_feature_key Only used if use_deprecated_input_fn is false. String key into the features dict returned by input_fn that corresponds to a the raw Example strings Tensor that the exported model will take as input. Can only be None if you're using a custom signature_fn that does not use the first arg (examples).
    use_deprecated_input_fn Determines the signature format of input_fn.
    signature_fn Function that returns a default signature and a named signature map, given Tensor of Example strings, dict of Tensors for features and Tensor or dict of Tensors for predictions.
    prediction_key The key for a tensor in the predictions dict (output from the model_fn) to use as the predictions input to the signature_fn. Optional. If None, predictions will pass to signature_fn without filtering.
    default_batch_size Default batch size of the Example placeholder.
    exports_to_keep Number of exports to keep.
    checkpoint_path the checkpoint path of the model to be exported. If it is None (which is default), will use the latest checkpoint in export_dir.

    Returns
    The string path to the exported directory. NB: this functionality was added ca. 2016/09/25; clients that depend on the return value may need to handle the case where this function returns None because subclasses are not returning a value.

    export_savedmodel

    View source

    Exports inference graph as a SavedModel into given dir.

    Args
    export_dir_base A string containing a directory to write the exported graph and checkpoints.
    serving_input_fn A function that takes no argument and returns an InputFnOps.
    default_output_alternative_key the name of the head to serve when none is specified. Not needed for single-headed models.
    assets_extra A dict specifying how to populate the assets.extra directory within the exported SavedModel. Each key should give the destination path (including the filename) relative to the assets.extra directory. The corresponding value gives the full path of the source file to be copied. For example, the simple case of copying a single file without renaming it is specified as {'my_asset_file.txt': '/path/to/my_asset_file.txt'}.
    as_text whether to write the SavedModel proto in text format.
    checkpoint_path The checkpoint path to export. If None (the default), the most recent checkpoint found within the model directory is chosen.
    graph_rewrite_specs an iterable of GraphRewriteSpec. Each element will produce a separate MetaGraphDef within the exported SavedModel, tagged and rewritten as specified. Defaults to a single entry using the default serving tag ("serve") and no rewriting.
    strip_default_attrs Boolean. If True, default-valued attributes will be removed from the NodeDefs. For a detailed guide, see Stripping Default-Valued Attributes.

    Returns
    The string path to the exported directory.

    Raises
    ValueError if an unrecognized export_type is requested.

    fit

    View source

    See Trainable. (deprecated arguments)

    Raises
    ValueError If x or y are not None while input_fn is not None.
    ValueError If both steps and max_steps are not None.

    get_col_factors

    View source