tfx.components.trainer.executor.Executor

Local estimator based trainer executor used by the TFX Trainer component.

Inherits From: GenericExecutor, BaseExecutor

tfx.components.trainer.executor.Executor(
    context: Optional[tfx.dsl.components.base.base_executor.BaseExecutor.Context] = None
)

How to create a trainer callback function to be used by this Trainer executor: An estimator can be executed by TFX by first creating a trainer_fn callback method that returns an estimator and some additional parameters, similar to https://github.com/tensorflow/tfx/blob/master/tfx/examples/chicago_taxi_pipeline/taxi_utils.py#L285 This becomes the basis of the new Executor for Trainer. This Executor will then train and evaluate this estimator using the tf.estimator.train_and_evaluate API to train locally.

Child Classes

class Context

Methods

Do

View source

Do(
    input_dict: Dict[Text, List[types.Artifact]],
    output_dict: Dict[Text, List[types.Artifact]],
    exec_properties: Dict[Text, Any]
) -> None

Uses a user-supplied tf.estimator to train a TensorFlow model locally.

The Trainer Executor invokes a training_fn callback function provided by the user via the module_file parameter. With the tf.estimator returned by this function, the Trainer Executor then builds a TensorFlow model using the user-provided tf.estimator.

Args
input_dict	Input dict from input key to a list of ML-Metadata Artifacts. examples: Examples used for training, must include 'train' and 'eval' if custom splits is not specified in train_args and eval_args. transform_graph: Optional input transform graph. schema: Schema of the data.
output_dict	Output dict from output key to a list of Artifacts. model: Exported model. model_run: Model training related outputs (e.g., Tensorboard logs)
exec_properties	A dict of execution properties. train_args: JSON string of trainer_pb2.TrainArgs instance, providing args for training. eval_args: JSON string of trainer_pb2.EvalArgs instance, providing args for eval. module_file: Python module file containing UDF model definition. Exactly one of module_file, module_path and trainer_fn should be passed. module_path: Python module path containing UDF model definition. Exactly one of module_file, module_path and trainer_fn should be passed. trainer_fn: Python module path to the trainer function. Exactly one of module_file, module_path and trainer_fn should be passed. warm_starting: Whether or not we need to do warm starting. warm_start_from: Optional. If warm_starting is True, this is the directory to find previous model to warm start on. custom_config: Optional. JSON-serialized dict of additional parameters to pass to trainer function.

Returns
None

Raises
ValueError	When not exactly one of module_file, module_path and trainer_fn are present in exec_properties.