tfx.orchestration.launcher.kubernetes_component_launcher.KubernetesComponentLauncher

Responsible for launching a container executor on Kubernetes.

Inherits From: BaseComponentLauncher

tfx.orchestration.launcher.kubernetes_component_launcher.KubernetesComponentLauncher(
    component: tfx.dsl.components.base.base_node.BaseNode,
    pipeline_info: tfx.orchestration.data_types.PipelineInfo,
    driver_args: tfx.orchestration.data_types.DriverArgs,
    metadata_connection: tfx.orchestration.metadata.Metadata,
    beam_pipeline_args: List[Text],
    additional_pipeline_args: Dict[Text, Any],
    component_config: Optional[tfx.orchestration.config.base_component_config.BaseComponentConfig] = None
)

component The Tfx node to launch.
pipeline_info An instance of data_types.PipelineInfo that holds pipeline properties.
driver_args An instance of data_types.DriverArgs that holds component specific driver args.
metadata_connection ML metadata connection. The connection is expected to not be opened when given to this object.
beam_pipeline_args Pipeline arguments for Beam powered Components.
additional_pipeline_args Additional pipeline args.
component_config Optional component specific config to instrument launcher on how to launch a component.

ValueError when component and component_config are not launchable by the launcher.

Methods

can_launch

View source

@classmethod
can_launch(
    component_executor_spec: tfx.dsl.components.base.executor_spec.ExecutorSpec,
    component_config: tfx.orchestration.config.base_component_config.BaseComponentConfig = None
) -> bool

Checks if the launcher can launch the executor spec.

create

View source

@classmethod
create(
    component: tfx.dsl.components.base.base_node.BaseNode,
    pipeline_info: tfx.orchestration.data_types.PipelineInfo,
    driver_args: tfx.orchestration.data_types.DriverArgs,
    metadata_connection: tfx.orchestration.metadata.Metadata,
    beam_pipeline_args: List[Text],
    additional_pipeline_args: Dict[Text, Any],
    component_config: Optional[tfx.orchestration.config.base_component_config.BaseComponentConfig] = None
) -> "BaseComponentLauncher"

Initialize a ComponentLauncher directly from a BaseComponent instance.

This class method is the contract between TfxRunner and BaseComponentLauncher to support launcher polymorphism. Sublcass of this class must make sure it can be initialized by the method.

Args
component The component to launch.
pipeline_info An instance of data_types.PipelineInfo that holds pipeline properties.
driver_args An instance of data_types.DriverArgs that holds component specific driver args.
metadata_connection ML metadata connection. The connection is expected to not be opened when given to this object.
beam_pipeline_args Pipeline arguments for Beam powered Components.
additional_pipeline_args Additional pipeline args.
component_config Optional component specific config to instrument launcher on how to launch a component.

Returns
A new instance of component launcher.

launch

View source

launch() -> tfx.orchestration.data_types.ExecutionInfo

Execute the component, includes driver, executor and publisher.

Returns
The execution decision of the launch.