tfx.v1.orchestration.experimental.KubeflowV2DagRunner

Kubeflow V2 pipeline runner (currently for managed pipelines).

tfx.v1.orchestration.experimental.KubeflowV2DagRunner(
    config: tfx.v1.orchestration.experimental.KubeflowV2DagRunnerConfig,
    output_dir: Optional[str] = None,
    output_filename: Optional[str] = None
)

Used in the notebooks

Used in the tutorials
Reading data from BigQuery with TFX and Vertex Pipelines Simple TFX Pipeline for Vertex Pipelines Vertex AI Training and Serving with TFX and Vertex Pipelines

Builds a pipeline job spec in json format based on TFX pipeline DSL object.

Args
config	An KubeflowV2DagRunnerConfig object to specify runtime configuration when running the pipeline in Kubeflow.
output_dir	An optional output directory into which to output the pipeline definition files. Defaults to the current working directory.
output_filename	An optional output file name for the pipeline definition file. The file output format will be a JSON-serialized PipelineJob pb message. Defaults to 'pipeline.json'.

Methods

run

run(
    pipeline: tfx.v1.dsl.Pipeline,
    parameter_values: Optional[Dict[str, Any]] = None,
    write_out: Optional[bool] = True
) -> Dict[str, Any]

Compiles a pipeline DSL object into pipeline file.

Args
pipeline	TFX pipeline object.
parameter_values	mapping from runtime parameter names to its values.
write_out	set to True to actually write out the file to the place designated by output_dir and output_filename. Otherwise return the JSON-serialized pipeline job spec.

Returns
Returns the JSON pipeline job spec.

Raises
RuntimeError	if trying to write out to a place occupied by an existing file.

set_exit_handler

set_exit_handler(
    exit_handler: tfx.v1.types.BaseNode
)

Set exit handler components for the Kubeflow V2(Vertex AI) dag runner.

This feature is currently experimental without backward compatibility gaurantee.

Args
exit_handler	exit handler component.