cwltool.workflow_job

Attributes

CWLObjectType

Typical raw dictionary found in lightly parsed CWL.

CWLOutputType

JobsGeneratorType

OutputCallbackType

ParametersType

ScatterDestinationsType

ScatterOutputCallbackType

SinkType

Exceptions

WorkflowException

Common base class for all non-exit exceptions.

Classes

RuntimeContext

Shared kwargs based initializer for RuntimeContext and LoadingContext.

StdFsAccess

Local filesystem implementation.

WorkflowStateItem

Workflow state item.

WorkflowJobStep

Generated for each step in Workflow.steps().

ReceiveScatterOutput

Produced by the scatter generators.

WorkflowJob

Generates steps from the Workflow.

WorkflowJobLoopStep

Generated for each step in Workflow.steps() containing a Loop requirement.

Functions

content_limit_respected_read(f)

Read a file as a string, respecting the CONTENT_LIMIT.

can_assign_src_to_sink(src, sink[, strict])

Check for identical type specifications, ignoring extra keys like inputBinding.

getdefault(val, default)

Return the val using the default as backup in case the val is None.

shortname(inputid)

uniquename(stem[, names])

adjustDirObjs(rec, op)

Apply an update function to each Directory object in the object rec.

aslist(thing)

Wrap any non-MutableSequence/list in a list.

get_listing(fs_access, rec[, recursive])

Expand, recursively, any 'listing' fields in a Directory.

parallel_steps(steps, rc, runtimeContext)

nested_crossproduct_scatter(process, joborder, ...)

crossproduct_size(joborder, scatter_keys)

Compute the size of a cross product.

flat_crossproduct_scatter(process, joborder, ...)

dotproduct_scatter(process, joborder, scatter_keys, ...)

match_types(sinktype, src, iid, inputobj, linkMerge, ...)

object_from_state(state, params, frag_only, ...[, ...])

Module Contents

cwltool.workflow_job.content_limit_respected_read(f)

Read a file as a string, respecting the CONTENT_LIMIT.

Parameters:

f (IO[bytes]) – file handle

Returns:

the file contents

Raises:

WorkflowException – if the file is too large

Return type:

str

cwltool.workflow_job.can_assign_src_to_sink(src, sink, strict=False)

Check for identical type specifications, ignoring extra keys like inputBinding.

In non-strict comparison, at least one source type must match one sink type, except for ‘null’. In strict comparison, all source types must match at least one sink type.

Parameters:
  • src (cwltool.utils.SinkType) – admissible source types

  • sink (Optional[cwltool.utils.SinkType]) – admissible sink types

  • strict (bool)

Return type:

bool

class cwltool.workflow_job.RuntimeContext(kwargs=None)

Bases: ContextBase

Inheritance diagram of cwltool.workflow_job.RuntimeContext

Shared kwargs based initializer for RuntimeContext and LoadingContext.

Parameters:

kwargs (Optional[Dict[str, Any]])

outdir: str | None = None
tmpdir: str = ''
tmpdir_prefix: str
tmp_outdir_prefix: str = ''
stagedir: str = ''
get_outdir()

Return outdir or create one with tmp_outdir_prefix.

Return type:

str

get_tmpdir()

Return tmpdir or create one with tmpdir_prefix.

Return type:

str

get_stagedir()

Return stagedir or create one with tmpdir_prefix.

Return type:

str

create_tmpdir()

Create a temporary directory that respects tmpdir_prefix.

Return type:

str

create_outdir()

Create a temporary directory that respects tmp_outdir_prefix.

Return type:

str

copy()

Return a copy of this RuntimeContext.

Return type:

RuntimeContext

cwltool.workflow_job.getdefault(val, default)

Return the val using the default as backup in case the val is None.

Parameters:
  • val (Any)

  • default (Any)

Return type:

Any

exception cwltool.workflow_job.WorkflowException

Bases: Exception

Inheritance diagram of cwltool.workflow_job.WorkflowException

Common base class for all non-exit exceptions.

cwltool.workflow_job.shortname(inputid)
Parameters:

inputid (str)

Return type:

str

cwltool.workflow_job.uniquename(stem, names=None)
Parameters:
  • stem (str)

  • names (Optional[Set[str]])

Return type:

str

class cwltool.workflow_job.StdFsAccess(basedir)

Local filesystem implementation.

Parameters:

basedir (str)

glob(pattern)
Parameters:

pattern (str)

Return type:

List[str]

open(fn, mode)
Parameters:
Return type:

IO[Any]

exists(fn)
Parameters:

fn (str)

Return type:

bool

size(fn)
Parameters:

fn (str)

Return type:

int

isfile(fn)
Parameters:

fn (str)

Return type:

bool

isdir(fn)
Parameters:

fn (str)

Return type:

bool

listdir(fn)
Parameters:

fn (str)

Return type:

List[str]

join(path, *paths)
Parameters:
Return type:

str

realpath(path)
Parameters:

path (str)

Return type:

str

cwltool.workflow_job.CWLObjectType

Typical raw dictionary found in lightly parsed CWL.

cwltool.workflow_job.CWLOutputType
cwltool.workflow_job.JobsGeneratorType
cwltool.workflow_job.OutputCallbackType
cwltool.workflow_job.ParametersType
cwltool.workflow_job.ScatterDestinationsType
cwltool.workflow_job.ScatterOutputCallbackType
cwltool.workflow_job.SinkType
class cwltool.workflow_job.WorkflowStateItem

Bases: NamedTuple

Inheritance diagram of cwltool.workflow_job.WorkflowStateItem

Workflow state item.

parameter: CWLObjectType
value: CWLOutputType | None
success: str
cwltool.workflow_job.adjustDirObjs(rec, op)

Apply an update function to each Directory object in the object rec.

Parameters:
  • rec (Any)

  • op (Union[Callable[Ellipsis, Any], partial[Any]])

Return type:

None

cwltool.workflow_job.aslist(thing)

Wrap any non-MutableSequence/list in a list.

Parameters:

thing (Any)

Return type:

MutableSequence[Any]

cwltool.workflow_job.get_listing(fs_access, rec, recursive=True)

Expand, recursively, any ‘listing’ fields in a Directory.

Parameters:
Return type:

None

class cwltool.workflow_job.WorkflowJobStep(step)

Generated for each step in Workflow.steps().

Parameters:

step (cwltool.workflow.WorkflowStep)

job(joborder, output_callback, runtimeContext)
Parameters:
Return type:

cwltool.utils.JobsGeneratorType

class cwltool.workflow_job.ReceiveScatterOutput(output_callback, dest, total)

Produced by the scatter generators.

Parameters:
  • output_callback (cwltool.utils.ScatterOutputCallbackType)

  • dest (cwltool.utils.ScatterDestinationsType)

  • total (int)

receive_scatter_output(index, jobout, processStatus)

Record the results of a scatter operation.

Parameters:
  • index (int)

  • jobout (cwltool.utils.CWLObjectType)

  • processStatus (str)

Return type:

None

setTotal(total, steps)

Set the total number of expected outputs along with the steps.

This is necessary to finish the setup.

Parameters:
  • total (int)

  • steps (List[Optional[cwltool.utils.JobsGeneratorType]])

Return type:

None

cwltool.workflow_job.parallel_steps(steps, rc, runtimeContext)
Parameters:
Return type:

cwltool.utils.JobsGeneratorType

cwltool.workflow_job.nested_crossproduct_scatter(process, joborder, scatter_keys, output_callback, runtimeContext)
Parameters:
Return type:

cwltool.utils.JobsGeneratorType

cwltool.workflow_job.crossproduct_size(joborder, scatter_keys)

Compute the size of a cross product.

Parameters:
  • joborder (cwltool.utils.CWLObjectType)

  • scatter_keys (MutableSequence[str])

Return type:

int

cwltool.workflow_job.flat_crossproduct_scatter(process, joborder, scatter_keys, output_callback, runtimeContext)
Parameters:
Return type:

cwltool.utils.JobsGeneratorType

cwltool.workflow_job.dotproduct_scatter(process, joborder, scatter_keys, output_callback, runtimeContext)
Parameters:
Return type:

cwltool.utils.JobsGeneratorType

cwltool.workflow_job.match_types(sinktype, src, iid, inputobj, linkMerge, valueFrom)
Parameters:
  • sinktype (Optional[cwltool.utils.SinkType])

  • src (cwltool.utils.WorkflowStateItem)

  • iid (str)

  • inputobj (cwltool.utils.CWLObjectType)

  • linkMerge (Optional[str])

  • valueFrom (Optional[str])

Return type:

bool

cwltool.workflow_job.object_from_state(state, params, frag_only, supportsMultipleInput, sourceField, incomplete=False)
Parameters:
  • state (Dict[str, Optional[cwltool.utils.WorkflowStateItem]])

  • params (cwltool.utils.ParametersType)

  • frag_only (bool)

  • supportsMultipleInput (bool)

  • sourceField (str)

  • incomplete (bool)

Return type:

Optional[cwltool.utils.CWLObjectType]

class cwltool.workflow_job.WorkflowJob(workflow, runtimeContext)

Generates steps from the Workflow.

Parameters:
do_output_callback(final_output_callback)
Parameters:

final_output_callback (cwltool.utils.OutputCallbackType)

Return type:

None

receive_output(step, outputparms, final_output_callback, jobout, processStatus)
Parameters:
  • step (WorkflowJobStep)

  • outputparms (List[cwltool.utils.CWLObjectType])

  • final_output_callback (cwltool.utils.OutputCallbackType)

  • jobout (cwltool.utils.CWLObjectType)

  • processStatus (str)

Return type:

None

try_make_job(step, final_output_callback, runtimeContext)
Parameters:
Return type:

cwltool.utils.JobsGeneratorType

run(runtimeContext, tmpdir_lock=None)

Log the start of each workflow.

Parameters:
Return type:

None

job(joborder, output_callback, runtimeContext)
Parameters:
Return type:

cwltool.utils.JobsGeneratorType

class cwltool.workflow_job.WorkflowJobLoopStep(step, container_engine)

Generated for each step in Workflow.steps() containing a Loop requirement.

Parameters:
job(joborder, output_callback, runtimeContext)

Generate a WorkflowJobStep job until the loopWhen condition evaluates to False.

Parameters:
Return type:

cwltool.utils.JobsGeneratorType

loop_callback(runtimeContext, jobout, processStatus)

Update the joborder object with output values from the last iteration.

Parameters:
Return type:

None