Kubeflow pipelines task builder.
tfx.orchestration.kubeflow.v2.step_builder.StepBuilder(
node: tfx.dsl.components.base.base_node.BaseNode
,
deployment_config: pipeline_pb2.PipelineDeploymentConfig,
image: Optional[Text] = None,
image_cmds: Optional[List[Text]] = None,
beam_pipeline_args: Optional[List[Text]] = None,
enable_cache: bool = False,
pipeline_info: Optional[tfx.orchestration.data_types.PipelineInfo
] = None,
channel_redirect_map: Optional[Dict[Tuple[Text, Text], Text]] = None
)
Constructs a pipeline task spec based on the TFX node information. Meanwhile,
augments the deployment config associated with the node.
Args |
node
|
A TFX node. The logical unit of a step. Note, currently for resolver
node we only support two types of resolver
policies, including: 1) latest blessed model, and 2) latest model
artifact.
|
deployment_config
|
The deployment config in Kubeflow IR to be populated.
|
image
|
TFX image used in the underlying container spec. Required if node
is a TFX component.
|
image_cmds
|
Optional. If not specified the default ENTRYPOINT defined
in the docker image will be used. Note: the commands here refers to the
K8S container command, which maps to Docker entrypoint field. If one
supplies command but no args are provided for the container, the
container will be invoked with the provided command, ignoring the
ENTRYPOINT and CMD defined in the Dockerfile. One can find more
details regarding the difference between K8S and Docker conventions at
https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#notes
|
beam_pipeline_args
|
Pipeline arguments for Beam powered Components.
|
enable_cache
|
If true, enables cache lookup for this pipeline step.
Defaults to False.
|
pipeline_info
|
Optionally, the pipeline info associated with current
pipeline. The pipeline context is required if the current node is a
resolver. Defaults to None.
|
channel_redirect_map
|
Map from (producer component id, output key) to (new
producer component id, output key). This is needed for cases where one
DSL node is splitted into multiple tasks in pipeline API proto. For
example, latest blessed model resolver.
|
Raises |
ValueError
|
On the following two cases:
- The node being built is an instance of BaseComponent but image was
not provided.
- The node being built is a Resolver but the associated pipeline
info was not provided.
|
Methods
build
View source
build() -> List[pipeline_pb2.PipelineTaskSpec]
Builds a pipeline StepSpec given the node information.
Returns |
A list of PipelineTaskSpec messages corresponding to the node. For most of
the cases, the list contains a single element. The only exception is when
compiling latest blessed model resolver. One DSL node will be
split to two resolver specs to reflect the two-phased query execution.
|
Raises |
NotImplementedError
|
When the node being built is an InfraValidator.
|