tfx.orchestration.beam.legacy.beam_dag_runner.BeamDagRunner

Tfx runner on Beam.

Inherits From: TfxRunner

tfx.orchestration.beam.legacy.beam_dag_runner.BeamDagRunner(
    beam_orchestrator_args: Optional[List[Text]] = None,
    config: Optional[tfx.orchestration.config.pipeline_config.PipelineConfig] = None
)

Args
beam_orchestrator_args	beam args for the beam orchestrator. Note that this is different from the beam_pipeline_args within additional_pipeline_args, which is for beam pipelines in components.
config	Optional pipeline config for customizing the launching of each component. Defaults to pipeline config that supports InProcessComponentLauncher and DockerComponentLauncher.

Methods

run

View source

run(
    tfx_pipeline: tfx.orchestration.pipeline.Pipeline
) -> None

Deploys given logical pipeline on Beam.

Args
tfx_pipeline	Logical pipeline containing pipeline args and components.