tfx.orchestration.portable.launcher.Launcher

Launcher is the main entrance of nodes in TFleX.

tfx.orchestration.portable.launcher.Launcher(
    pipeline_node: pipeline_pb2.PipelineNode,
    mlmd_connection: tfx.orchestration.metadata.Metadata,
    pipeline_info: pipeline_pb2.PipelineInfo,
    pipeline_runtime_spec: pipeline_pb2.PipelineRuntimeSpec,
    executor_spec: Optional[message.Message] = None,
    custom_driver_spec: Optional[message.Message] = None,
    platform_spec: Optional[message.Message] = None,
    custom_executor_operators: Optional[Dict[Any, Type[tfx.orchestration.portable.launcher.ExecutorOperator]]] = None,
    custom_driver_operators: Optional[Dict[Any, Type[tfx.orchestration.portable.launcher.DriverOperator]]] = None
)

It handles TFX internal details like artifact resolving, execution triggering and result publishing.

pipeline_node The specification of the node that this launcher lauches.
mlmd_connection ML metadata connection.
pipeline_info The information of the pipeline that this node runs in.
pipeline_runtime_spec The runtime information of the pipeline that this node runs in.
executor_spec Specification for the executor of the node. This is expected for all components nodes. This will be used to determine the specific ExecutorOperator class to be used to execute and will be passed into ExecutorOperator.
custom_driver_spec Specification for custom driver. This is expected only for advanced use cases.
platform_spec Platform config that will be used as auxiliary info of the node execution. This will be passed to ExecutorOperator along with the executor_spec.
custom_executor_operators a map of ExecutableSpec to its ExecutorOperation implementation.
custom_driver_operators a map of ExecutableSpec to its DriverOperator implementation.

ValueError when component and component_config are not launchable by the launcher.

Methods

launch

View source

launch() -> Optional[metadata_store_pb2.Execution]

Executes the component, includes driver, executor and publisher.

Returns
The metadata of this execution that is registered in MLMD. It can be None if the driver decides not to run the execution.