tf.keras.utils.SequenceEnqueuer

TensorFlow 1 version View source on GitHub

Class SequenceEnqueuer

Base class to enqueue inputs.

The task of an Enqueuer is to use parallelism to speed up preprocessing. This is done with processes or threads.

Example:

    enqueuer = SequenceEnqueuer(...)
    enqueuer.start()
    datas = enqueuer.get()
    for data in datas:
        # Use the inputs; training, evaluating, predicting.
        # ... stop sometime.
    enqueuer.close()

The enqueuer.get() should be an infinite stream of datas.

__init__

View source

__init__(
    sequence,
    use_multiprocessing=False
)

Initialize self. See help(type(self)) for accurate signature.

Methods

get

View source

get()

Creates a generator to extract data from the queue.

Skip the data if it is None.

Returns

Generator yielding tuples `(inputs, targets)`
    or `(inputs, targets, sample_weights)`.

is_running

View source

is_running()

start

View source

start(
    workers=1,
    max_queue_size=10
)

Starts the handler's workers.

Arguments:

  • workers: Number of workers.
  • max_queue_size: queue size (when full, workers could block on put())

stop

View source

stop(timeout=None)

Stops running threads and wait for them to exit, if necessary.

Should be called by the same thread which called start().

Arguments:

  • timeout: maximum time to wait on thread.join()

Compat aliases