tfx.components.trainer.executor.Executor

Local estimator based trainer executor used by the TFX Trainer component.

Inherits From: GenericExecutor, BaseExecutor

tfx.components.trainer.executor.Executor(
    context: Optional[tfx.dsl.components.base.base_executor.BaseExecutor.Context] = None
)

How to create a trainer callback function to be used by this Trainer executor: An estimator can be executed by TFX by first creating a trainer_fn callback method that returns an estimator and some additional parameters, similar to https://github.com/tensorflow/tfx/blob/master/tfx/examples/chicago_taxi_pipeline/taxi_utils.py#L285 This becomes the basis of the new Executor for Trainer. This Executor will then train and evaluate this estimator using the tf.estimator.train_and_evaluate API to train locally.

Child Classes

class Context

Methods

Do

View source

Do(
    input_dict: Dict[Text, List[types.Artifact]],
    output_dict: Dict[Text, List[types.Artifact]],
    exec_properties: Dict[Text, Any]
) -> None

Uses a user-supplied tf.estimator to train a TensorFlow model locally.

The Trainer Executor invokes a training_fn callback function provided by the user via the module_file parameter. With the tf.estimator returned by this function, the Trainer Executor then builds a TensorFlow model using the user-provided tf.estimator.

Args
input_dict Input dict from input key to a list of ML-Metadata Artifacts.

  • examples: Examples used for training, must include 'train' and 'eval' if custom splits is not specified in train_args and eval_args.
  • transform_output: Optional input transform graph.
  • schema: Schema of the data.
output_dict Output dict from output key to a list of Artifacts.
  • model: Exported model.
  • model_run: Model training related outputs (e.g., Tensorboard logs)
    • exec_properties A dict of execution properties.
  • train_args: JSON string of trainer_pb2.TrainArgs instance, providing args for training.
  • eval_args: JSON string of trainer_pb2.EvalArgs instance, providing args for eval.
  • module_file: Python module file containing UDF model definition.
  • warm_starting: Whether or not we need to do warm starting.
  • warm_start_from: Optional. If warm_starting is True, this is the directory to find previous model to warm start on.
  • custom_config: Optional. JSON-serialized dict of additional parameters to pass to trainer function.

    • Returns
    None

    Raises
    ValueError When neither or both of 'module_file' and 'trainer_fn' are present in 'exec_properties'.