tfx.orchestration.experimental.core.sync_pipeline_task_gen.SyncPipelineTaskGenerator

Task generator for executing a sync pipeline.

Inherits From: TaskGenerator

tfx.orchestration.experimental.core.sync_pipeline_task_gen.SyncPipelineTaskGenerator(
    mlmd_handle: tfx.orchestration.metadata.Metadata,
    pipeline: pipeline_pb2.Pipeline,
    is_task_id_tracked_fn: Callable[[task_lib.TaskId], bool]
)

Calling generate is not thread-safe. Concurrent calls to generate should be explicitly serialized. Since MLMD may be updated upon call to generate, it's also not safe to call generate on different instances of this class where the instances refer to the same MLMD db and the same pipeline IR.

Args
mlmd_handle	A handle to the MLMD db.
pipeline	A pipeline IR proto.
is_task_id_tracked_fn	A callable that returns True if a task_id is tracked by the task queue.

Methods

generate

View source

generate() -> List[tfx.orchestration.experimental.core.task.Task]

Generates tasks for executing the next executable nodes in the pipeline.

The returned tasks must have exec_task populated. List may be empty if no nodes are ready for execution.

Returns
A list of tasks to execute.