View source on GitHub
|
TaskGenerator interface.
When their generate method is invoked (typically done periodically within an
orchestration loop), concrete classes implementing this interface are expected
to generate tasks to execute nodes in a pipeline IR spec or system tasks (eg:
for garbage collection) based on the state of pipeline execution and related
details stored in an MLMD db.
Note on thread safety: Concrete classes of this interface need not have a
thread-safe implementation. Onus is on the caller to serialize concurrent
calls to generate. Since MLMD db may be updated upon call to generate,
it's also not safe to invoke generate concurrently on different instances
of TaskGenerator that refer to the same MLMD db and the same pipeline IR.
Methods
generate
@abc.abstractmethodgenerate() -> List[tfx.orchestration.experimental.core.task.Task]
Generates a list of tasks to be performed.
| Returns | |
|---|---|
A list of Tasks specifying nodes in a pipeline to be executed or other
system tasks.
|