Join the SIG TFX-Addons community and help make TFX even better!

tfx.orchestration.kubeflow.v2.kubeflow_v2_dag_runner.KubeflowV2DagRunner

Kubeflow V2 pipeline runner.

Inherits From: TfxRunner

Builds a pipeline job spec in json format based on TFX pipeline DSL object.

config An KubeflowV2DagRunnerConfig object to specify runtime configuration when running the pipeline in Kubeflow.
output_dir An optional output directory into which to output the pipeline definition files. Defaults to the current working directory.
output_filename An optional output file name for the pipeline definition file. The file output format will be a JSON-serialized PipelineJob pb message. Defaults to 'pipeline.json'.

Methods

compile

View source

Compiles a pipeline DSL object into pipeline file.

Args
pipeline TFX pipeline object.
parameter_values mapping from runtime parameter names to its values.
write_out set to True to actually write out the file to the place designated by output_dir and output_filename. Otherwise return the JSON-serialized pipeline job spec.

Returns
Returns the JSON pipeline job spec.

Raises
RuntimeError if trying to write out to a place occupied by an existing file.

run

View source

Compiles a pipeline DSL object into pipeline file.

Args
pipeline TFX pipeline object.
parameter_values mapping from runtime parameter names to its values.
write_out set to True to actually write out the file to the place designated by output_dir and output_filename. Otherwise return the JSON-serialized pipeline job spec.

Returns
Returns the JSON pipeline job spec.

Raises
RuntimeError if trying to write out to a place occupied by an existing file.