Base class to enqueue inputs.
The task of an Enqueuer is to use parallelism to speed up preprocessing. This is done with processes or threads.
enqueuer = SequenceEnqueuer(...) enqueuer.start() datas = enqueuer.get() for data in datas: # Use the inputs; training, evaluating, predicting. # ... stop sometime. enqueuer.close()
enqueuer.get() should be an infinite stream of datas.
Creates a generator to extract data from the queue.
Skip the data if it is
Generator yielding tuples
(inputs, targets, sample_weights).
start( workers=1, max_queue_size=10 )
Starts the handler's workers.
workers: number of worker threads
max_queue_size: queue size (when full, threads could block on
Stop running threads and wait for them to exit, if necessary.
Should be called by the same thread which called start().
timeout: maximum time to wait on thread.join()