![]() |
A thread-safe task queue with duplicate detection.
tfx.orchestration.experimental.core.task_queue.TaskQueue()
The life-cycle of a task starts with producers calling enqueue
. Consumers
call dequeue
to obtain the tasks in FIFO order. When processing is complete,
consumers must release the tasks by calling task_done
.
Methods
contains_task_id
contains_task_id(
task_id: task_lib.TaskId
) -> bool
Returns True
if the task queue contains a task with the given task_id
.
Args | |
---|---|
task_id
|
A task id. |
Returns | |
---|---|
True if a task with task_id was enqueued but task_done has not been
invoked yet.
|
dequeue
dequeue(
max_wait_secs: Optional[float] = None
) -> Optional[tfx.orchestration.experimental.core.task.Task
]
Removes and returns a task from the queue.
Once the processing is complete, queue consumers must call task_done
.
Args | |
---|---|
max_wait_secs
|
If not None , waits a maximum of max_wait_secs when the
queue is empty for a task to be enqueued. If no task is present in the
queue after the wait, None is returned. If max_wait_secs is None
(default), returns None without waiting when the queue is empty.
|
Returns | |
---|---|
A Task or None if the queue is empty.
|
enqueue
enqueue(
task: tfx.orchestration.experimental.core.task.Task
) -> bool
Enqueues the given task if no prior task with the same id exists.
Args | |
---|---|
task
|
A Task object.
|
Returns | |
---|---|
True if the task could be enqueued. False if a task with the same id
already exists.
|
is_empty
is_empty() -> bool
Returns True
if the task queue is empty.
Queue is considered empty only if any enqueued tasks have been dequeued and
task_done
invoked on them.
task_done
task_done(
task: tfx.orchestration.experimental.core.task.Task
) -> None
Marks the processing of a task as done.
Consumers should call this method after the task is processed.
Args | |
---|---|
task
|
A Task object.
|
Raises | |
---|---|
RuntimeError
|
If attempt is made to mark a non-existent or non-dequeued task as done. |