cwltool.process
¶
Classes and methods relevant for all CWL Process types.
Module Contents¶
Classes¶
Filter instances are used to perform arbitrary filtering of LogRecords. |
|
Base class for get_requirement(). |
Functions¶
|
|
|
|
|
|
|
|
|
|
|
Link or copy files to their targets. Create them as needed. |
|
|
|
|
|
|
|
|
|
Add missing information to a type so that CWL types are valid. |
|
|
|
Detect any textual reference to /var/spool/cwl. |
|
|
|
|
|
|
|
|
|
Given a CWL document or input object, search for dependencies |
|
Attributes¶
- class cwltool.process.LogAsDebugFilter(name, parent)¶
Bases:
logging.Filter
Filter instances are used to perform arbitrary filtering of LogRecords.
Loggers and Handlers can optionally use Filter instances to filter records as desired. The base filter class only allows events which are below a certain point in the logger hierarchy. For example, a filter initialized with “A.B” will allow events logged by loggers “A.B”, “A.B.C”, “A.B.C.D”, “A.B.D” etc. but not “A.BB”, “B.A.B” etc. If initialized with the empty string, all events are passed.
- Parameters
name (str) –
parent (logging.Logger) –
- filter(self, record)¶
Determine if the specified record is to be logged.
Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place.
- Parameters
record (logging.LogRecord) –
- Return type
bool
- cwltool.process.supportedProcessRequirements¶
- cwltool.process.cwl_files = ['Workflow.yml', 'CommandLineTool.yml', 'CommonWorkflowLanguage.yml', 'Process.yml',...¶
- cwltool.process.salad_files = ['metaschema.yml', 'metaschema_base.yml', 'salad.md', 'field_name.yml', 'import_include.md',...¶
- cwltool.process.SCHEMA_CACHE :Dict[str, Tuple[Loader, Union[Names, SchemaParseException], CWLObjectType, Loader]]¶
- cwltool.process.SCHEMA_FILE :Optional[CWLObjectType]¶
- cwltool.process.SCHEMA_DIR :Optional[CWLObjectType]¶
- cwltool.process.SCHEMA_ANY :Optional[CWLObjectType]¶
- cwltool.process.custom_schemas :Dict[str, Tuple[str, str]]¶
- cwltool.process.use_standard_schema(version)¶
- Parameters
version (str) –
- Return type
None
- cwltool.process.use_custom_schema(version, name, text)¶
- Parameters
version (str) –
name (str) –
text (str) –
- Return type
None
- cwltool.process.get_schema(version)¶
- Parameters
version (str) –
- Return type
Tuple[schema_salad.ref_resolver.Loader, Union[schema_salad.avro.schema.Names, schema_salad.avro.schema.SchemaParseException], cwltool.utils.CWLObjectType, schema_salad.ref_resolver.Loader]
- cwltool.process.shortname(inputid)¶
- Parameters
inputid (str) –
- Return type
str
- cwltool.process.checkRequirements(rec, supported_process_requirements)¶
- Parameters
rec (Union[MutableSequence[cwltool.utils.CWLObjectType], cwltool.utils.CWLObjectType, cwltool.utils.CWLOutputType, None]) –
supported_process_requirements (Iterable[str]) –
- Return type
None
- cwltool.process.stage_files(pathmapper, stage_func=None, ignore_writable=False, symlink=True, secret_store=None, fix_conflicts=False)¶
Link or copy files to their targets. Create them as needed.
- Parameters
pathmapper (cwltool.pathmapper.PathMapper) –
stage_func (Optional[Callable[[str, str], None]]) –
ignore_writable (bool) –
symlink (bool) –
secret_store (Optional[cwltool.secrets.SecretStore]) –
fix_conflicts (bool) –
- Return type
None
- cwltool.process.relocateOutputs(outputObj, destination_path, source_directories, action, fs_access, compute_checksum=True, path_mapper=PathMapper)¶
- Parameters
outputObj (cwltool.utils.CWLObjectType) –
destination_path (str) –
source_directories (Set[str]) –
action (str) –
fs_access (cwltool.stdfsaccess.StdFsAccess) –
compute_checksum (bool) –
path_mapper (Type[cwltool.pathmapper.PathMapper]) –
- Return type
cwltool.utils.CWLObjectType
- cwltool.process.cleanIntermediate(output_dirs)¶
- Parameters
output_dirs (Iterable[str]) –
- Return type
None
- cwltool.process.add_sizes(fsaccess, obj)¶
- Parameters
fsaccess (cwltool.stdfsaccess.StdFsAccess) –
obj (cwltool.utils.CWLObjectType) –
- Return type
None
- cwltool.process.fill_in_defaults(inputs, job, fsaccess)¶
- Parameters
inputs (List[cwltool.utils.CWLObjectType]) –
job (cwltool.utils.CWLObjectType) –
fsaccess (cwltool.stdfsaccess.StdFsAccess) –
- Return type
None
- cwltool.process.avroize_type(field_type, name_prefix='')¶
Add missing information to a type so that CWL types are valid.
- Parameters
field_type (Union[cwltool.utils.CWLObjectType, MutableSequence[Any], cwltool.utils.CWLOutputType, None]) –
name_prefix (str) –
- Return type
Union[cwltool.utils.CWLObjectType, MutableSequence[Any], cwltool.utils.CWLOutputType, None]
- cwltool.process.get_overrides(overrides, toolid)¶
- Parameters
overrides (MutableSequence[cwltool.utils.CWLObjectType]) –
toolid (str) –
- Return type
cwltool.utils.CWLObjectType
- cwltool.process.var_spool_cwl_detector(obj, item=None, obj_key=None)¶
Detect any textual reference to /var/spool/cwl.
- Parameters
obj (cwltool.utils.CWLOutputType) –
item (Optional[Any]) –
obj_key (Optional[Any]) –
- Return type
bool
- cwltool.process.eval_resource(builder, resource_req)¶
- Parameters
builder (cwltool.builder.Builder) –
resource_req (Union[str, int, float]) –
- Return type
Optional[Union[str, int, float]]
- cwltool.process.FILE_COUNT_WARNING = 5000¶
- class cwltool.process.Process(toolpath_object, loadingContext)¶
Bases:
cwltool.utils.HasReqsHints
Base class for get_requirement().
- Parameters
toolpath_object (ruamel.yaml.comments.CommentedMap) –
loadingContext (cwltool.context.LoadingContext) –
- evalResources(self, builder, runtimeContext)¶
- Parameters
builder (cwltool.builder.Builder) –
runtimeContext (cwltool.context.RuntimeContext) –
- Return type
Dict[str, Union[int, float]]
- validate_hints(self, avsc_names, hints, strict)¶
- Parameters
avsc_names (schema_salad.avro.schema.Names) –
hints (List[cwltool.utils.CWLObjectType]) –
strict (bool) –
- Return type
None
- visit(self, op)¶
- Parameters
op (Callable[[ruamel.yaml.comments.CommentedMap], None]) –
- Return type
None
- abstract job(self, job_order, output_callbacks, runtimeContext)¶
- Parameters
job_order (cwltool.utils.CWLObjectType) –
output_callbacks (Optional[cwltool.utils.OutputCallbackType]) –
runtimeContext (cwltool.context.RuntimeContext) –
- Return type
cwltool.utils.JobsGeneratorType
- __str__(self)¶
Return the id of this CWL process.
- Return type
str
- cwltool.process.uniquename(stem, names=None)¶
- Parameters
stem (str) –
names (Optional[Set[str]]) –
- Return type
str
- cwltool.process.nestdir(base, deps)¶
- Parameters
base (str) –
deps (cwltool.utils.CWLObjectType) –
- Return type
cwltool.utils.CWLObjectType
- cwltool.process.mergedirs(listing)¶
- Parameters
listing (MutableSequence[cwltool.utils.CWLObjectType]) –
- Return type
MutableSequence[cwltool.utils.CWLObjectType]
- cwltool.process.CWL_IANA = https://www.iana.org/assignments/media-types/application/cwl¶
- cwltool.process.scandeps(base, doc, reffields, urlfields, loadref, urljoin=urllib.parse.urljoin, nestdirs=True)¶
Given a CWL document or input object, search for dependencies (references to external files) of ‘doc’ and return them as a list of File or Directory objects.
The ‘base’ is the base URL for relative references.
Looks for objects with ‘class: File’ or ‘class: Directory’ and adds them to the list of dependencies.
Anything in ‘urlfields’ is also added as a File dependency.
Anything in ‘reffields’ (such as workflow step ‘run’) will be added as a dependency and also loaded (using the ‘loadref’ function) and recursively scanned for dependencies. Those dependencies will be added as secondary files to the primary file.
If “nestdirs” is true, create intermediate directory objects when a file is located in a subdirectory under the starting directory. This is so that if the dependencies are materialized, they will produce the same relative file system locations.
- Parameters
base (str) –
doc (Union[cwltool.utils.CWLObjectType, MutableSequence[cwltool.utils.CWLObjectType]]) –
reffields (Set[str]) –
urlfields (Set[str]) –
loadref (Callable[[str, str], Union[ruamel.yaml.comments.CommentedMap, ruamel.yaml.comments.CommentedSeq, str, None]]) –
urljoin (Callable[[str, str], str]) –
nestdirs (bool) –
- Return type
MutableSequence[cwltool.utils.CWLObjectType]
- cwltool.process.compute_checksums(fs_access, fileobj)¶
- Parameters
fs_access (cwltool.stdfsaccess.StdFsAccess) –
fileobj (cwltool.utils.CWLObjectType) –
- Return type
None