cwltool.task_queue

TaskQueue.

Module Contents

Classes

TaskQueue

A TaskQueue class.

class cwltool.task_queue.TaskQueue(lock, thread_count)

A TaskQueue class.

Uses a first-in, first-out queue of tasks executed on a fixed number of threads.

New tasks enter the queue and are started in the order received, as worker threads become available.

If thread_count == 0 then tasks will be synchronously executed when add() is called (this makes the actual task queue behavior a no-op, but may be a useful configuration knob).

The thread_count is also used as the maximum size of the queue.

The threads are created during TaskQueue initialization. Call join() when you’re done with the TaskQueue and want the threads to stop.

Parameters:

• lock (threading.Lock) –

• thread_count (int) –

in_flight: int = 0

The number of tasks in the queue.

add(task, unlock=None, check_done=None)

Add your task to the queue.

The optional unlock will be released prior to attempting to add the task to the queue.

If the optional “check_done” threading.Event’s flag is set, then we will skip adding this task to the queue.

If the TaskQueue was created with thread_count == 0 then your task will be synchronously executed.

Parameters:

• task (Callable[[], None]) –

• unlock (Optional[threading.Condition]) –

• check_done (Optional[threading.Event]) –

Return type:

None

drain()

Drain the queue.

Return type:

None

join()

Wait for all threads to complete.

Return type:

None