snakemake package

Subpackages

• snakemake.remote package
  • Submodules
  • snakemake.remote.EGA module
  • snakemake.remote.FTP module
  • snakemake.remote.GS module
  • snakemake.remote.HTTP module
  • snakemake.remote.NCBI module
  • snakemake.remote.S3 module
  • snakemake.remote.S3Mocked module
  • snakemake.remote.SFTP module
  • snakemake.remote.XRootD module
  • snakemake.remote.dropbox module
  • snakemake.remote.gfal module
  • snakemake.remote.gridftp module
  • snakemake.remote.iRODS module
  • snakemake.remote.webdav module
  • Module contents
• snakemake.report package
  • Module contents

Submodules

snakemake.benchmark module

snakemake.checkpoints module

class snakemake.checkpoints.Checkpoint(rule, checkpoints)

Bases: object

checkpoints

get(**wildcards)

rule

class snakemake.checkpoints.CheckpointJob(rule, output)

Bases: object

output

rule

class snakemake.checkpoints.Checkpoints

Bases: object

A namespace for checkpoints so that they can be accessed via dot notation.

register(rule)

snakemake.common module

class snakemake.common.Mode

Bases: object

Enum for execution mode of Snakemake. This handles the behavior of e.g. the logger.

cluster = 2

default = 0

subprocess = 1

class snakemake.common.TBDInt

Bases: int

An integer that prints into <TBD>

snakemake.common.get_container_image()

snakemake.common.get_last_stable_version()

snakemake.common.get_uuid(name)

class snakemake.common.lazy_property(method)

Bases: property

cached

static clean(instance, method)

method

snakemake.common.log_location(msg)

snakemake.common.num_if_possible(s)

Convert string to number if possible, otherwise return string.

snakemake.common.strip_prefix(text, prefix)

snakemake.conda module

snakemake.cwl module

snakemake.cwl.cwl(path, basedir, input, output, params, wildcards, threads, resources, log, config, rulename, use_singularity, bench_record, jobid)

Load cwl from the given basedir + path and execute it.

snakemake.cwl.dag_to_cwl(dag)

Convert a given DAG to a CWL workflow, which is returned as JSON object.

snakemake.cwl.job_to_cwl(job, dag, outputs, inputs)

Convert a job with its dependencies to a CWL workflow step.

snakemake.dag module

class snakemake.dag.Batch(rulename: str, idx: int, batches: int)

Bases: object

Definition of a batch for calculating only a partial DAG.

get_batch(items: list)

Return the defined batch of the given items. Items are usually input files.

is_final

class snakemake.dag.DAG(workflow, rules=None, dryrun=False, targetfiles=None, targetrules=None, forceall=False, forcerules=None, forcefiles=None, priorityfiles=None, priorityrules=None, untilfiles=None, untilrules=None, omitfiles=None, omitrules=None, ignore_ambiguity=False, force_incomplete=False, ignore_incomplete=False, notemp=False, keep_remote_local=False, batch=None)

Bases: object

Directed acyclic graph of jobs.

archive(path)

Archives workflow such that it can be re-run on a different system.

Archiving includes git versioned files (i.e. Snakefiles, config files, …), ancestral input files and conda environments.

bfs(direction, *jobs, stop=<function DAG.<lambda>>)

Perform a breadth-first traversal of the DAG.

cache_job(job)

check_and_touch_output(job, wait=3, ignore_missing_output=False, no_touch=False, force_stay_on_remote=False)

Raise exception if output files of job are missing.

check_directory_outputs()

Check that no output file is contained in a directory output of the same or another rule.

check_dynamic()

Check dynamic output and update downstream rules if necessary.

check_incomplete()

Check if any output files are incomplete. This is done by looking up markers in the persistence module.

check_periodic_wildcards(job)

Raise an exception if a wildcard of the given job appears to be periodic, indicating a cyclic dependency.

checkpoint_jobs

clean(only_temp=False, dryrun=False)

Removes files generated by the workflow.

cleanup()

close_remote_objects()

Close all remote objects.

collect_potential_dependencies(job)

Collect all potential dependencies of a job. These might contain ambiguities. The keys of the returned dict represent the files to be considered.

create_conda_envs(dryrun=False, forceall=False, init_only=False, quiet=False)

d3dag(max_jobs=10000)

delete_job(job, recursive=True, add_dependencies=False)

Delete given job from DAG.

delete_omitfrom_jobs()

Removes jobs downstream of jobs specified by –omit-from.

dfs(direction, *jobs, stop=<function DAG.<lambda>>, post=True)

Perform depth-first traversal of the DAG.

dot()

downstream_of_omitfrom()

Returns the downstream of –omit-from rules or files and themselves.

dynamic(job)

Return whether a job is dynamic (i.e. it is only a placeholder for those that are created after the job with dynamic output has finished.

dynamic_output_jobs

Iterate over all jobs with dynamic output files.

file2jobs(targetfile)

filegraph_dot(node2rule=<function DAG.<lambda>>, node2style=<function DAG.<lambda>>, node2label=<function DAG.<lambda>>)

finish(job, update_dynamic=True)

Finish a given job (e.g. remove from ready jobs, mark depending jobs as ready).

finished(job)

Return whether a job is finished.

finished_jobs

Iterate over all jobs that have been finished.

get_jobs_or_groups()

handle_log(job, upload_remote=True)

handle_pipes()

Use pipes to determine job groups. Check if every pipe has exactly one consumer

handle_protected(job)

Write-protect output files that are marked with protected().

handle_remote(job, upload=True)

Remove local files if they are no longer needed and upload.

handle_temp(job)

Remove temp files if they are no longer needed. Update temp_mtimes.

handle_touch(job)

Touches those output files that are marked for touching.

in_omitfrom(job)

Return whether given job has been specified via –omit-from.

in_until(job)

Return whether given job has been specified via –until.

incomplete_external_jobid(job)

Return the external jobid of the job if it is marked as incomplete.

Returns None, if job is not incomplete, or if no external jobid has been registered or if force_incomplete is True.

incomplete_files

Return list of incomplete files.

init(progress=False)

Initialise the DAG.

is_batch_rule(rule)

Return True if the underlying rule is to be used for batching the DAG.

jobid(job)

Return job id of given job.

jobs

All jobs in the DAG.

level_bfs(direction, *jobs, stop=<function DAG.<lambda>>)

Perform a breadth-first traversal of the DAG, but also yield the level together with each job.

list_untracked()

List files in the workdir that are not in the dag.

local_needrun_jobs

Iterate over all jobs that need to be run and are marked as local.

missing_temp(job)

Return whether a temp file that is input of the given job is missing.

needrun(job)

Return whether a given job needs to be executed.

needrun_jobs

Jobs that need to be executed.

new_job(rule, targetfile=None, format_wildcards=None)

Create new job for given rule and (optional) targetfile. This will reuse existing jobs with the same wildcards.

new_wildcards(job)

Return wildcards that are newly introduced in this job, compared to its ancestors.

newversion_files

Return list of files where the current version is newer than the recorded version.

noneedrun_finished(job)

Return whether a given job is finished or was not required to run at all.

omitfrom_jobs()

Returns a generator of jobs specified by omitfromjobs.

postprocess()

Postprocess the DAG. This has to be invoked after any change to the DAG topology.

priority(job)

Return priority of given job.

pull_container_imgs(dryrun=False, forceall=False, quiet=False)

ready_jobs

Jobs that are ready to execute.

reason(job)

Return the reason of the job execution.

replace_job(job, newjob, recursive=True)

Replace given job with new job.

requested_files(job)

Return the files a job requests.

rule2job(targetrule)

Generate a new job from a given rule.

rule_dot()

rule_dot2()

set_until_jobs()

Removes jobs downstream of jobs specified by –omit-from.

specialize_rule(rule, newrule)

Specialize the given rule by inserting newrule into the DAG.

stats()

summary(detailed=False)

temp_input(job)

temp_size(job)

Return the total size of temporary input files of the job. If none, return 0.

unshadow_output(job, only_log=False)

Move files from shadow directory to real output paths.

until_jobs()

Returns a generator of jobs specified by untiljobs.

update(jobs, file=None, visited=None, skip_until_dynamic=False, progress=False)

Update the DAG by adding given jobs and their dependencies.

update_(job, visited=None, skip_until_dynamic=False, progress=False)

Update the DAG by adding the given job and its dependencies.

update_checkpoint_dependencies(jobs=None)

Update dependencies of checkpoints.

update_checkpoint_outputs()

update_dynamic(job)

Update the DAG by evaluating the output of the given job that contains dynamic output files.

update_groups()

update_jobids()

update_needrun()

Update the information whether a job needs to be executed.

update_output_index()

Update the OutputIndex.

update_priority()

Update job priorities.

update_ready(jobs=None)

Update information whether a job is ready to execute.

Given jobs must be needrun jobs!

snakemake.decorators module

snakemake.decorators.dec_all_methods(decorator, prefix='test_')

snakemake.exceptions module

exception snakemake.exceptions.AmbiguousRuleException(filename, job_a, job_b, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.AzureFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.CacheMissException

Bases: Exception

exception snakemake.exceptions.ChildIOException(parent=None, child=None, wildcards=None, lineno=None, snakefile=None, rule=None)

Bases: snakemake.exceptions.WorkflowError

exception snakemake.exceptions.ClusterJobException(job_info, jobid)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.CreateCondaEnvironmentException(*args, lineno=None, snakefile=None, rule=None)

Bases: snakemake.exceptions.WorkflowError

exception snakemake.exceptions.CreateRuleException(message=None, include=None, lineno=None, snakefile=None, rule=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.CyclicGraphException(repeatedrule, file, rule=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.DropboxFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.FTPFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.HTTPFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.IOException(prefix, rule, files, include=None, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.IOFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.ImproperOutputException(rule, files, include=None, lineno=None, snakefile=None)

Bases: snakemake.exceptions.IOException

exception snakemake.exceptions.ImproperShadowException(rule, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.IncompleteCheckpointException(rule, targetfile)

Bases: Exception

exception snakemake.exceptions.IncompleteFilesException(files)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.InputFunctionException(msg, wildcards=None, lineno=None, snakefile=None, rule=None)

Bases: snakemake.exceptions.WorkflowError

exception snakemake.exceptions.MissingInputException(rule, files, include=None, lineno=None, snakefile=None)

Bases: snakemake.exceptions.IOException

exception snakemake.exceptions.MissingOutputException(message=None, include=None, lineno=None, snakefile=None, rule=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.MissingRuleException(file, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.NCBIFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.NoRulesException(lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.PeriodicWildcardError(message=None, include=None, lineno=None, snakefile=None, rule=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.ProtectedOutputException(rule, files, include=None, lineno=None, snakefile=None)

Bases: snakemake.exceptions.IOException

exception snakemake.exceptions.RemoteFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.RuleException(message=None, include=None, lineno=None, snakefile=None, rule=None)

Bases: Exception

Base class for exception occuring within the execution or definition of rules.

messages

exception snakemake.exceptions.S3FileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.SFTPFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.SpawnedJobError

Bases: Exception

exception snakemake.exceptions.TerminatedException

Bases: Exception

exception snakemake.exceptions.UnexpectedOutputException(rule, files, include=None, lineno=None, snakefile=None)

Bases: snakemake.exceptions.IOException

exception snakemake.exceptions.UnknownRuleException(name, prefix='', lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.WebDAVFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

exception snakemake.exceptions.WildcardError(*args, lineno=None, snakefile=None, rule=None)

Bases: snakemake.exceptions.WorkflowError

exception snakemake.exceptions.WorkflowError(*args, lineno=None, snakefile=None, rule=None)

Bases: Exception

static format_arg(arg)

exception snakemake.exceptions.XRootDFileException(msg, lineno=None, snakefile=None)

Bases: snakemake.exceptions.RuleException

snakemake.exceptions.cut_traceback(ex)

snakemake.exceptions.format_error(ex, lineno, linemaps=None, snakefile=None, show_traceback=False)

snakemake.exceptions.format_traceback(tb, linemaps)

snakemake.exceptions.get_exception_origin(ex, linemaps)

snakemake.exceptions.log_verbose_traceback(ex)

snakemake.exceptions.print_exception(ex, linemaps)

Print an error message for a given exception.

Arguments ex – the exception linemaps – a dict of a dict that maps for each snakefile

the compiled lines to source code lines in the snakefile.

snakemake.executors module

class snakemake.executors.AbstractExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, printthreads=True, latency_wait=3, keepincomplete=False)

Bases: object

cancel()

get_default_remote_provider_args()

get_default_resources_args()

handle_job_error(job)

handle_job_success(job)

print_job_error(job, msg=None, **kwargs)

printjob(job)

rule_prefix(job)

run(job, callback=None, submit_callback=None, error_callback=None)

shutdown()

class snakemake.executors.CPUExecutor(workflow, dag, workers, printreason=False, quiet=False, printshellcmds=False, use_threads=False, latency_wait=3, cores=1, keepincomplete=False)

Bases: snakemake.executors.RealExecutor

cached_or_run(job, run_func, *args)

Either retrieve result from cache, or run job with given function.

cancel()

handle_job_error(job)

handle_job_success(job)

job_args_and_prepare(job)

run(job, callback=None, submit_callback=None, error_callback=None)

run_group_job(job)

Run a pipe group job.

This lets all items run simultaneously.

run_single_job(job)

shutdown()

spawn_job(job)

class snakemake.executors.ClusterExecutor(workflow, dag, cores, jobname='snakejob.{name}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, cluster_config=None, local_input=None, restart_times=None, exec_job=None, assume_shared_fs=True, max_status_checks_per_second=1, disable_default_remote_provider_args=False, disable_get_default_resources_args=False, keepincomplete=False)

Bases: snakemake.executors.RealExecutor

cancel()

cluster_params(job)

Return wildcards object for job from cluster_config.

cluster_wildcards(job)

default_jobscript = 'jobscript.sh'

format_job(pattern, job, **kwargs)

get_jobscript(job)

handle_job_error(job)

handle_job_success(job)

print_cluster_job_error(job_info, jobid)

shutdown()

tmpdir

write_jobscript(job, jobscript, **kwargs)

class snakemake.executors.DRMAAClusterJob(job, jobid, callback, error_callback, jobscript)

Bases: tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobid

Alias for field number 1

jobscript

Alias for field number 4

class snakemake.executors.DRMAAExecutor(workflow, dag, cores, jobname='snakejob.{rulename}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, drmaa_args='', drmaa_log_dir=None, latency_wait=3, cluster_config=None, restart_times=0, assume_shared_fs=True, max_status_checks_per_second=1, keepincomplete=False)

Bases: snakemake.executors.ClusterExecutor

cancel()

run(job, callback=None, submit_callback=None, error_callback=None)

shutdown()

class snakemake.executors.DryrunExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, printthreads=True, latency_wait=3, keepincomplete=False)

Bases: snakemake.executors.AbstractExecutor

printcache(job)

printjob(job)

class snakemake.executors.GenericClusterExecutor(workflow, dag, cores, submitcmd='qsub', statuscmd=None, cluster_config=None, jobname='snakejob.{rulename}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, restart_times=0, assume_shared_fs=True, max_status_checks_per_second=1, keepincomplete=False)

Bases: snakemake.executors.ClusterExecutor

cancel()

register_job(job)

run(job, callback=None, submit_callback=None, error_callback=None)

class snakemake.executors.GenericClusterJob(job, jobid, callback, error_callback, jobscript, jobfinished, jobfailed)

Bases: tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobfailed

Alias for field number 6

jobfinished

Alias for field number 5

jobid

Alias for field number 1

jobscript

Alias for field number 4

class snakemake.executors.KubernetesExecutor(workflow, dag, namespace, container_image=None, jobname='{rulename}.{jobid}', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, cluster_config=None, local_input=None, restart_times=None, keepincomplete=False)

Bases: snakemake.executors.ClusterExecutor

cancel()

register_secret()

run(job, callback=None, submit_callback=None, error_callback=None)

shutdown()

unregister_secret()

class snakemake.executors.KubernetesJob(job, jobid, callback, error_callback, kubejob, jobscript)

Bases: tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobid

Alias for field number 1

jobscript

Alias for field number 5

kubejob

Alias for field number 4

class snakemake.executors.RealExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, latency_wait=3, assume_shared_fs=True, keepincomplete=False)

Bases: snakemake.executors.AbstractExecutor

format_job_pattern(pattern, job=None, **kwargs)

handle_job_error(job, upload_remote=True)

handle_job_success(job, upload_remote=True, handle_log=True, handle_touch=True, ignore_missing_output=False)

register_job(job)

class snakemake.executors.SynchronousClusterExecutor(workflow, dag, cores, submitcmd='qsub', cluster_config=None, jobname='snakejob.{rulename}.{jobid}.sh', printreason=False, quiet=False, printshellcmds=False, latency_wait=3, restart_times=0, assume_shared_fs=True, keepincomplete=False)

Bases: snakemake.executors.ClusterExecutor

invocations like “qsub -sync y” (SGE) or “bsub -K” (LSF) are synchronous, blocking the foreground thread and returning the remote exit code at remote exit.

cancel()

run(job, callback=None, submit_callback=None, error_callback=None)

class snakemake.executors.SynchronousClusterJob(job, jobid, callback, error_callback, jobscript, process)

Bases: tuple

callback

Alias for field number 2

error_callback

Alias for field number 3

job

Alias for field number 0

jobid

Alias for field number 1

jobscript

Alias for field number 4

process

Alias for field number 5

class snakemake.executors.TibannaExecutor(workflow, dag, cores, tibanna_sfn, precommand='', tibanna_config=False, container_image=None, printreason=False, quiet=False, printshellcmds=False, latency_wait=3, local_input=None, restart_times=None, max_status_checks_per_second=1, keepincomplete=False)

Bases: snakemake.executors.ClusterExecutor

add_command(job, tibanna_args, tibanna_config)

add_workflow_files(job, tibanna_args)

adjust_filepath(f)

cancel()

handle_remote(target)

make_tibanna_input(job)

remove_prefix(s)

run(job, callback=None, submit_callback=None, error_callback=None)

shutdown()

split_filename(filename, checkdir=None)

class snakemake.executors.TibannaJob(job, jobname, jobid, exec_arn, callback, error_callback)

Bases: tuple

callback

Alias for field number 4

error_callback

Alias for field number 5

exec_arn

Alias for field number 3

job

Alias for field number 0

jobid

Alias for field number 2

jobname

Alias for field number 1

class snakemake.executors.TouchExecutor(workflow, dag, printreason=False, quiet=False, printshellcmds=False, latency_wait=3, assume_shared_fs=True, keepincomplete=False)

Bases: snakemake.executors.RealExecutor

handle_job_success(job)

run(job, callback=None, submit_callback=None, error_callback=None)

snakemake.executors.change_working_directory(directory=None)

Change working directory in execution context if provided.

snakemake.executors.run_wrapper(job_rule, input, output, params, wildcards, threads, resources, log, benchmark, benchmark_repeats, conda_env, container_img, singularity_args, env_modules, use_singularity, linemaps, debug, cleanup_scripts, shadow_dir, jobid)

Wrapper around the run method that handles exceptions and benchmarking.

Arguments job_rule – the job.rule member input – list of input files output – list of output files wildcards – so far processed wildcards threads – usable threads log – list of log files shadow_dir – optional shadow directory root

snakemake.executors.sleep()

snakemake.gui module

snakemake.io module

class snakemake.io.AnnotatedString(value)

Bases: str

class snakemake.io.IOCache

Bases: object

clear()

deactivate()

snakemake.io.IOFile(file, rule=None)

class snakemake.io.InputFiles(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)

Bases: snakemake.io.Namedlist

size

size_mb

class snakemake.io.Log(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)

Bases: snakemake.io.Namedlist

class snakemake.io.Namedlist(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)

Bases: list

A list that additionally provides functions to name items. Further, it is hashable, however the hash does not consider the item names.

get(key, default_value=None)

items()

keys()

class snakemake.io.OutputFiles(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)

Bases: snakemake.io.Namedlist

class snakemake.io.Params(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)

Bases: snakemake.io.Namedlist

class snakemake.io.PeriodicityDetector(min_repeat=20, max_repeat=100)

Bases: object

is_periodic(value)

Returns the periodic substring or None if not periodic.

class snakemake.io.ReportObject(caption, category, subcategory, patterns)

Bases: tuple

caption

Alias for field number 0

category

Alias for field number 1

patterns

Alias for field number 3

subcategory

Alias for field number 2

class snakemake.io.Resources(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)

Bases: snakemake.io.Namedlist

class snakemake.io.Wildcards(toclone=None, fromdict=None, plainstr=False, strip_constraints=False, custom_map=None)

Bases: snakemake.io.Namedlist

snakemake.io.ancient(value)

A flag for an input file that shall be considered ancient; i.e. its timestamp shall have no effect on which jobs to run.

snakemake.io.apply_wildcards(pattern, wildcards, fill_missing=False, fail_dynamic=False, dynamic_fill=None, keep_dynamic=False)

snakemake.io.checkpoint_target(value)

snakemake.io.contains_wildcard(path)

snakemake.io.contains_wildcard_constraints(pattern)

snakemake.io.directory(value)

A flag to specify that an output is a directory, rather than a file or named pipe.

snakemake.io.dynamic(value)

A flag for a file that shall be dynamic, i.e. the multiplicity (and wildcard values) will be expanded after a certain rule has been run

snakemake.io.expand(*args, **wildcards)

Expand wildcards in given filepatterns.

Arguments *args – first arg: filepatterns as list or one single filepattern,

second arg (optional): a function to combine wildcard values (itertools.product per default)

**wildcards – the wildcards as keyword arguments

with their values as lists. If allow_missing=True is included wildcards in filepattern without values will stay unformatted.

snakemake.io.flag(value, flag_type, flag_value=True)

snakemake.io.get_flag_value(value, flag_type)

snakemake.io.get_git_root(path)

Parameters:path – (str) Path a to a directory/file that is located inside the repo Returns:path to root folder for git repo

snakemake.io.get_git_root_parent_directory(path, input_path)

This function will recursively go through parent directories until a git repository is found or until no parent directories are left, in which case a error will be raised. This is needed when providing a path to a file/folder that is located on a branch/tag no currently checked out.

Parameters:

• path – (str) Path a to a directory that is located inside the repo
• input_path – (str) origin path, used when raising WorkflowError

Returns:

path to root folder for git repo

snakemake.io.get_wildcard_names(pattern)

snakemake.io.git_content(git_file)

This function will extract a file from a git repository, one located on the filesystem. Expected format is git+file:///path/to/your/repo/path_to_file@@version

Parameters:env_file (str) – consist of path to repo, @, version and file information Ex: git+file:////home/smeds/snakemake-wrappers/bio/fastqc/wrapper.py@0.19.3 Returns:file content or None if the expected format isn’t meet

snakemake.io.glob_wildcards(pattern, files=None, followlinks=False)

Glob the values of the wildcards by matching the given pattern to the filesystem. Returns a named tuple with a list of values for each wildcard.

snakemake.io.is_callable(value)

snakemake.io.is_flagged(value, flag)

snakemake.io.lchmod(f, mode)

snakemake.io.limit(pattern, **wildcards)

Limit wildcards to the given values.

Arguments: **wildcards – the wildcards as keyword arguments

with their values as lists

snakemake.io.load_configfile(configpath)

Loads a JSON or YAML configfile as a dict, then checks that it’s a dict.

snakemake.io.local(value)

Mark a file as local file. This disables application of a default remote provider.

snakemake.io.lutime(f, times)

snakemake.io.multiext(prefix, *extensions)

Expand a given prefix with multiple extensions (e.g. .txt, .csv, …).

snakemake.io.not_iterable(value)

snakemake.io.pipe(value)

snakemake.io.protected(value)

A flag for a file that shall be write protected after creation.

snakemake.io.regex(filepattern)

snakemake.io.remove(file, remove_non_empty_dir=False)

snakemake.io.repeat(value, n_repeat)

Flag benchmark records with the number of repeats.

snakemake.io.report(value, caption=None, category=None, subcategory=None, patterns=[])

Flag output file or directory as to be included into reports.

In case of directory, files to include can be specified via a glob pattern (default: *).

Arguments value – File or directory. caption – Path to a .rst file with a textual description of the result. category – Name of the category in which the result should be displayed in the report. pattern – Wildcard pattern for selecting files if a directory is given (this is used as

input for snakemake.io.glob_wildcards). Pattern shall not include the path to the directory itself.

snakemake.io.split_git_path(path)

snakemake.io.strip_wildcard_constraints(pattern)

Return a string that does not contain any wildcard constraints.

snakemake.io.temp(value)

A flag for an input or output file that shall be removed after usage.

snakemake.io.temporary(value)

An alias for temp.

snakemake.io.touch(value)

snakemake.io.unpack(value)

snakemake.io.update_wildcard_constraints(pattern, wildcard_constraints, global_wildcard_constraints)

Update wildcard constraints

Parameters:

• pattern (str) – pattern on which to update constraints
• wildcard_constraints (dict) – dictionary of wildcard:constraint key-value pairs
• global_wildcard_constraints (dict) – dictionary of wildcard:constraint key-value pairs

snakemake.io.wait_for_files(files, latency_wait=3, force_stay_on_remote=False, ignore_pipe=False)

Wait for given files to be present in filesystem.

snakemake.jobs module

class snakemake.jobs.AbstractJob

Bases: object

download_remote_input()

is_group()

log_error(msg=None, **kwargs)

log_info(skip_dynamic=False)

properties(omit_resources=['_cores', '_nodes'], **aux_properties)

remove_existing_output()

class snakemake.jobs.GroupJob(id, jobs)

Bases: snakemake.jobs.AbstractJob

all_products

attempt

check_protected_output()

cleanup()

dag

download_remote_input()

dynamic_input

expanded_output

Yields the entire expanded output of all jobs

finalize()

format_wildcards(string, **variables)

Format a string with variables from the job.

get_targets()

get_wait_for_files()

groupid

input

inputsize

is_branched

is_checkpoint

is_group()

is_local

is_updated

jobid

jobs

log

log_error(msg=None, **kwargs)

log_info(skip_dynamic=False)

merge(other)

name

needs_singularity

output

postprocess(error=False, **kwargs)

priority

products

properties(omit_resources=['_cores', '_nodes'], **aux_properties)

register()

remove_existing_output()

resources

restart_times

rules

threads

toposorted

class snakemake.jobs.Job(rule, dag, wildcards_dict=None, format_wildcards=None, targetfile=None)

Bases: snakemake.jobs.AbstractJob

HIGHEST_PRIORITY = 9223372036854775807

archive_conda_env()

Archive a conda environment into a custom local channel.

attempt

b64id

benchmark

benchmark_repeats

check_protected_output()

cleanup()

Cleanup output files.

close_remote()

conda_env

conda_env_file

conda_env_path

container_img

container_img_path

container_img_url

dag

dependencies

download_remote_input()

dynamic_input

dynamic_output

dynamic_wildcards

Return all wildcard values determined from dynamic output.

empty_remote_dirs

env_modules

existing_output

existing_remote_input

existing_remote_output

expand_dynamic(pattern)

Expand dynamic files.

expanded_output

Iterate over output files while dynamic output is expanded.

files_to_download

files_to_upload

format_wildcards(string, **variables)

Format a string with variables from the job.

get_targets()

get_wait_for_files()

group

input

input_maxtime

Return newest input file.

inputsize

Return the size of the input files. Input files need to be present.

is_branched

is_checkpoint

is_cwl

is_group()

is_local

is_norun

is_notebook

is_pipe

is_run

is_script

is_shadow

is_shell

is_valid()

Check if job is valid

is_wrapper

jobid

local_input

local_output

log

log_error(msg=None, indent=False, **kwargs)

log_info(skip_dynamic=False, indent=False, printshellcmd=True)

message

Return the message for this job.

missing_input

Return missing input files.

missing_output(requested=None)

Return missing output files.

missing_remote_input

missing_remote_output

name

needs_singularity

output

output_mintime

Return oldest output file.

output_mintime_local

outputs_older_than_script_or_notebook()

return output that’s older than script, i.e. script has changed

params

postprocess(upload_remote=True, handle_log=True, handle_touch=True, handle_temp=True, error=False, ignore_missing_output=False, assume_shared_fs=True, latency_wait=None)

prepare()

Prepare execution of job. This includes creation of directories and deletion of previously created dynamic files. Creates a shadow directory for the job if specified.

priority

products

properties(omit_resources=['_cores', '_nodes'], **aux_properties)

protected_output

register()

remote_input

remote_input_newer_than_local

remote_input_older_than_local

remote_output

remote_output_newer_than_local

remote_output_older_than_local

remove_existing_output()

Clean up both dynamic and regular output before rules actually run

resources

restart_times

rmdir_empty_remote_dirs()

rule

rules

shadow_dir

shadowed_path(f)

Get the shadowed path of IOFile f.

shellcmd

Return the shell command.

subworkflow_input

targetfile

temp_output

threads

touch_output

unique_input

updated()

wildcards

wildcards_dict

class snakemake.jobs.Reason

Bases: object

derived

forced

incomplete_output

missing_output

noio

nooutput

pipe

updated_input

updated_input_run

snakemake.jobs.format_files(job, io, dynamicio)

snakemake.jobs.jobfiles(jobs, type)

snakemake.logging module

class snakemake.logging.ColorizingStreamHandler(nocolor=False, stream=<_io.TextIOWrapper name='<stderr>' mode='w' encoding='UTF-8'>, use_threads=False, mode=0)

Bases: logging.StreamHandler

BLACK = 0

BLUE = 4

BOLD_SEQ = '\x1b[1m'

COLOR_SEQ = '\x1b[%dm'

CYAN = 6

GREEN = 2

MAGENTA = 5

RED = 1

RESET_SEQ = '\x1b[0m'

WHITE = 7

YELLOW = 3

can_color_tty(mode)

colors = {'CRITICAL': 1, 'DEBUG': 4, 'ERROR': 1, 'INFO': 2, 'WARNING': 3}

decorate(record)

emit(record)

Emit a record.

If a formatter is specified, it is used to format the record. The record is then written to the stream with a trailing newline. If exception information is present, it is formatted using traceback.print_exception and appended to the stream. If the stream has an ‘encoding’ attribute, it is used to determine how to do the output to the stream.

is_tty

class snakemake.logging.Logger

Bases: object

cleanup()

d3dag(**msg)

dag_debug(msg)

debug(msg)

error(msg)

get_logfile()

group_error(**msg)

group_info(**msg)

handler(msg)

info(msg, indent=False)

job_error(**msg)

job_finished(**msg)

job_info(**msg)

location(msg)

logfile_hint()

progress(done=None, total=None)

remove_logfile()

resources_info(msg)

rule_info(**msg)

run_info(msg)

set_level(level)

set_stream_handler(stream_handler)

setup_logfile()

shellcmd(msg, indent=False)

text_handler(msg)

The default snakemake log handler.

Prints the output to the console.

Parameters:msg (dict) – the log message dictionary

warning(msg)

class snakemake.logging.SlackLogger

Bases: object

log_handler(msg)

snakemake.logging.format_dict(dict_like, omit_keys=[], omit_values=[])

snakemake.logging.format_resource_names(resources, omit_resources=['_cores', '_nodes'])

snakemake.logging.format_resources(dict_like, *, omit_keys={'_cores', '_nodes'}, omit_values=[])

snakemake.logging.format_wildcards(dict_like, omit_keys=[], *, omit_values={'__snakemake_dynamic__'})

snakemake.logging.setup_logger(handler=[], quiet=False, printshellcmds=False, printreason=False, debug_dag=False, nocolor=False, stdout=False, debug=False, use_threads=False, mode=0, show_failed_logs=False)

snakemake.output_index module

class snakemake.output_index.OutputIndex(rules)

Bases: object

match(targetfile)

snakemake.parser module

class snakemake.parser.AbstractCmd(snakefile, rulename, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.Run

args()

block_content(token)

decorate_end(token)

end()

end_func = None

is_block_end(token)

overwrite_block_content(token)

overwrite_cmd = None

start()

start_func = None

class snakemake.parser.Benchmark(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.CWL(snakefile, rulename, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.Script

args()

end_func = 'cwl'

start_func = '@workflow.cwl'

class snakemake.parser.Cache(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

keyword

class snakemake.parser.Checkpoint(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.Rule

start()

class snakemake.parser.Conda(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Configfile(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

class snakemake.parser.Container(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.DecoratorKeywordState(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.KeywordState

args = []

decorator = None

end()

start()

class snakemake.parser.EnvModules(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Envvars(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

keyword

class snakemake.parser.GlobalContainer(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

keyword

class snakemake.parser.GlobalKeywordState(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.KeywordState

start()

class snakemake.parser.GlobalSingularity(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

keyword

class snakemake.parser.GlobalWildcardConstraints(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

keyword

class snakemake.parser.Group(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Include(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

class snakemake.parser.Input(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.KeywordState(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.TokenAutomaton

block(token)

block_content(token)

colon(token)

decorate_end(token)

end()

is_block_end(token)

keyword

prefix = ''

yield_indent(token)

class snakemake.parser.Localrules(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

block_content(token)

class snakemake.parser.Log(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Message(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Notebook(snakefile, rulename, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.Script

end_func = 'notebook'

start_func = '@workflow.notebook'

class snakemake.parser.OnError(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.DecoratorKeywordState

args = ['log']

decorator = 'onerror'

class snakemake.parser.OnStart(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.DecoratorKeywordState

args = ['log']

decorator = 'onstart'

class snakemake.parser.OnSuccess(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.DecoratorKeywordState

args = ['log']

decorator = 'onsuccess'

class snakemake.parser.Output(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Params(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Priority(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Python(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.TokenAutomaton

python(token)

subautomata = {'checkpoint': <class 'snakemake.parser.Checkpoint'>, 'configfile': <class 'snakemake.parser.Configfile'>, 'container': <class 'snakemake.parser.GlobalContainer'>, 'envvars': <class 'snakemake.parser.Envvars'>, 'include': <class 'snakemake.parser.Include'>, 'localrules': <class 'snakemake.parser.Localrules'>, 'onerror': <class 'snakemake.parser.OnError'>, 'onstart': <class 'snakemake.parser.OnStart'>, 'onsuccess': <class 'snakemake.parser.OnSuccess'>, 'report': <class 'snakemake.parser.Report'>, 'rule': <class 'snakemake.parser.Rule'>, 'ruleorder': <class 'snakemake.parser.Ruleorder'>, 'singularity': <class 'snakemake.parser.GlobalSingularity'>, 'subworkflow': <class 'snakemake.parser.Subworkflow'>, 'wildcard_constraints': <class 'snakemake.parser.GlobalWildcardConstraints'>, 'workdir': <class 'snakemake.parser.Workdir'>}

class snakemake.parser.Report(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

class snakemake.parser.Resources(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Rule(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

block_content(token)

dedent

end()

name(token)

start(aux='')

subautomata = {'benchmark': <class 'snakemake.parser.Benchmark'>, 'cache': <class 'snakemake.parser.Cache'>, 'conda': <class 'snakemake.parser.Conda'>, 'container': <class 'snakemake.parser.Container'>, 'cwl': <class 'snakemake.parser.CWL'>, 'envmodules': <class 'snakemake.parser.EnvModules'>, 'group': <class 'snakemake.parser.Group'>, 'input': <class 'snakemake.parser.Input'>, 'log': <class 'snakemake.parser.Log'>, 'message': <class 'snakemake.parser.Message'>, 'notebook': <class 'snakemake.parser.Notebook'>, 'output': <class 'snakemake.parser.Output'>, 'params': <class 'snakemake.parser.Params'>, 'priority': <class 'snakemake.parser.Priority'>, 'resources': <class 'snakemake.parser.Resources'>, 'run': <class 'snakemake.parser.Run'>, 'script': <class 'snakemake.parser.Script'>, 'shadow': <class 'snakemake.parser.Shadow'>, 'shell': <class 'snakemake.parser.Shell'>, 'singularity': <class 'snakemake.parser.Singularity'>, 'threads': <class 'snakemake.parser.Threads'>, 'version': <class 'snakemake.parser.Version'>, 'wildcard_constraints': <class 'snakemake.parser.WildcardConstraints'>, 'wrapper': <class 'snakemake.parser.Wrapper'>}

class snakemake.parser.RuleKeywordState(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.KeywordState

start()

class snakemake.parser.Ruleorder(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

block_content(token)

class snakemake.parser.Run(snakefile, rulename, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.RuleKeywordState

block_content(token)

end()

is_block_end(token)

start()

class snakemake.parser.Script(snakefile, rulename, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.AbstractCmd

args()

end_func = 'script'

start_func = '@workflow.script'

class snakemake.parser.Shadow(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.Shell(snakefile, rulename, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.AbstractCmd

args()

end_func = 'shell'

start_func = '@workflow.shellcmd'

class snakemake.parser.Singularity(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

keyword

class snakemake.parser.Snakefile(path, rulecount=0)

Bases: object

exception snakemake.parser.StopAutomaton(token)

Bases: Exception

class snakemake.parser.Subworkflow(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

block_content(token)

end()

name(token)

subautomata = {'configfile': <class 'snakemake.parser.SubworkflowConfigfile'>, 'snakefile': <class 'snakemake.parser.SubworkflowSnakefile'>, 'workdir': <class 'snakemake.parser.SubworkflowWorkdir'>}

class snakemake.parser.SubworkflowConfigfile(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.SubworkflowKeywordState

class snakemake.parser.SubworkflowKeywordState(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.KeywordState

end()

prefix = 'Subworkflow'

start()

class snakemake.parser.SubworkflowSnakefile(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.SubworkflowKeywordState

class snakemake.parser.SubworkflowWorkdir(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.SubworkflowKeywordState

class snakemake.parser.Threads(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.TokenAutomaton(snakefile, base_indent=0, dedent=0, root=True)

Bases: object

consume()

dedent

effective_indent

error(msg, token)

indentation(token)

subautomata = {}

subautomaton(automaton, *args, **kwargs)

class snakemake.parser.Version(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

class snakemake.parser.WildcardConstraints(snakefile, base_indent=0, dedent=0, root=True, rulename=None)

Bases: snakemake.parser.RuleKeywordState

keyword

class snakemake.parser.Workdir(snakefile, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.GlobalKeywordState

class snakemake.parser.Wrapper(snakefile, rulename, base_indent=0, dedent=0, root=True)

Bases: snakemake.parser.Script

args()

end_func = 'wrapper'

start_func = '@workflow.wrapper'

snakemake.parser.format_tokens(tokens)

snakemake.parser.is_colon(token)

snakemake.parser.is_comma(token)

snakemake.parser.is_comment(token)

snakemake.parser.is_dedent(token)

snakemake.parser.is_eof(token)

snakemake.parser.is_greater(token)

snakemake.parser.is_indent(token)

snakemake.parser.is_name(token)

snakemake.parser.is_newline(token, newline_tokens={4, 58})

snakemake.parser.is_op(token)

snakemake.parser.is_string(token)

snakemake.parser.lineno(token)

snakemake.parser.parse(path, overwrite_shellcmd=None, rulecount=0)

snakemake.persistence module

class snakemake.persistence.Persistence(nolock=False, dag=None, conda_prefix=None, singularity_prefix=None, shadow_prefix=None, warn_only=False)

Bases: object

all_inputfiles()

all_outputfiles()

cleanup(job)

cleanup_locks()

cleanup_metadata(path)

cleanup_shadow()

code(path)

code_changed(job, file=None)

Yields output files with changed code of bool if file given.

conda_cleanup_envs()

deactivate_cache()

external_jobids(job)

files

finished(job)

incomplete(job)

input(path)

input_changed(job, file=None)

Yields output files with changed input of bool if file given.

lock()

lock_warn_only()

locked

log(path)

metadata(path)

noop(*args)

params(path)

params_changed(job, file=None)

Yields output files with changed params of bool if file given.

rule(path)

shellcmd(path)

started(job, external_jobid=None)

unlock(*args)

version(path)

version_changed(job, file=None)

Yields output files with changed versions of bool if file given.

snakemake.persistence.pickle_code(code)

snakemake.rules module

class snakemake.rules.Rule(*args, lineno=None, snakefile=None, restart_times=0)

Bases: object

apply_default_remote(item)

apply_input_function(func, wildcards, incomplete_checkpoint_func=<function Rule.<lambda>>, raw_exceptions=False, **aux_params)

benchmark

check_caching()

check_output_duplicates()

Check Namedlist for duplicate entries and raise a WorkflowError on problems.

check_wildcards(wildcards)

conda_env

container_img

dynamic_branch(wildcards, input=True)

expand_benchmark(wildcards)

expand_conda_env(wildcards)

expand_group(wildcards)

Expand the group given wildcards.

expand_input(wildcards)

expand_log(wildcards)

expand_output(wildcards)

expand_params(wildcards, input, output, resources, omit_callable=False)

expand_resources(wildcards, input, attempt)

static get_wildcard_len(wildcards)

Return the length of the given wildcard values.

Arguments wildcards – a dict of wildcards

get_wildcards(requested_output)

Return wildcard dictionary by matching regular expression output files to the requested concrete ones.

Arguments requested_output – a concrete filepath

has_wildcards()

Return True if rule contains wildcards.

input

is_cwl

is_notebook

is_producer(requested_output)

Returns True if this rule is a producer of the requested output.

is_run

is_script

is_shell

is_wrapper

log

output

params

products

register_wildcards(wildcard_names)

set_input(*input, **kwinput)

Add a list of input files. Recursive lists are flattened.

Arguments input – the list of input files

set_log(*logs, **kwlogs)

set_output(*output, **kwoutput)

Add a list of output files. Recursive lists are flattened.

After creating the output files, they are checked for duplicates.

Arguments output – the list of output files

set_params(*params, **kwparams)

set_wildcard_constraints(**kwwildcard_constraints)

update_wildcard_constraints()

version

wildcard_constraints

wildcard_names

class snakemake.rules.RuleProxy(rule)

Bases: object

benchmark

input

log

output

params

class snakemake.rules.Ruleorder

Bases: object

add(*rulenames)

Records the order of given rules as rule1 > rule2 > rule3, …

compare(rule1, rule2)

Return whether rule2 has a higher priority than rule1.

snakemake.scheduler module

class snakemake.scheduler.DummyRateLimiter

Bases: contextlib.ContextDecorator

class snakemake.scheduler.JobScheduler(workflow, dag, cores, local_cores=1, dryrun=False, touch=False, cluster=None, cluster_status=None, cluster_config=None, cluster_sync=None, drmaa=None, drmaa_log_dir=None, kubernetes=None, container_image=None, tibanna=None, tibanna_sfn=None, precommand='', tibanna_config=False, jobname=None, quiet=False, printreason=False, printshellcmds=False, keepgoing=False, max_jobs_per_second=None, max_status_checks_per_second=100, latency_wait=3, greediness=1.0, force_use_threads=False, assume_shared_fs=True, keepincomplete=False)

Bases: object

calc_resource(name, value)

candidate(job)

Return whether a job is a candidate to be executed.

exit_gracefully(*args)

get_executor(job)

job_reward(job)

job_selector(jobs)

Using the greedy heuristic from “A Greedy Algorithm for the General Multidimensional Knapsack

Problem”, Akcay, Li, Xu, Annals of Operations Research, 2012

Args:

jobs (list): list of jobs

job_weight(job)

open_jobs

Return open jobs.

progress()

Display the progress.

rule_weight(rule)

run(job)

schedule()

Schedule jobs that are ready, maximizing cpu usage.

stats

snakemake.scheduler.cumsum(iterable, zero=[0])

snakemake.script module

class snakemake.script.JuliaEncoder

Bases: object

Encoding Pyton data structures into Julia.

classmethod encode_dict(d)

classmethod encode_items(items)

classmethod encode_list(l)

classmethod encode_namedlist(namedlist)

classmethod encode_positional_items(namedlist)

classmethod encode_value(value)

class snakemake.script.JuliaScript(path, source, basedir, input_, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir)

Bases: snakemake.script.ScriptBase

execute_script(fname)

get_preamble()

write_script(preamble, fd)

class snakemake.script.PythonScript(path, source, basedir, input_, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir)

Bases: snakemake.script.ScriptBase

execute_script(fname)

static generate_preamble(path, source, basedir, input_, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir, preamble_addendum='')

get_preamble()

write_script(preamble, fd)

class snakemake.script.REncoder

Bases: object

Encoding Pyton data structures into R.

classmethod encode_dict(d)

classmethod encode_items(items)

classmethod encode_list(l)

classmethod encode_namedlist(namedlist)

classmethod encode_numeric(value)

classmethod encode_value(value)

class snakemake.script.RMarkdown(path, source, basedir, input_, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir)

Bases: snakemake.script.ScriptBase

execute_script(fname)

get_preamble()

write_script(preamble, fd)

class snakemake.script.RScript(path, source, basedir, input_, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir)

Bases: snakemake.script.ScriptBase

execute_script(fname)

static generate_preamble(path, source, basedir, input_, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir, preamble_addendum='')

get_preamble()

write_script(preamble, fd)

class snakemake.script.ScriptBase(path, source, basedir, input_, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir)

Bases: abc.ABC

evaluate()

execute_script(fname)

get_preamble()

write_script(preamble, fd)

class snakemake.script.Snakemake(input_, output, params, wildcards, threads, resources, log, config, rulename, bench_iteration, scriptdir=None)

Bases: object

log_fmt_shell(stdout=True, stderr=True, append=False)

Return a shell redirection string to be used in shell() calls

This function allows scripts and wrappers support optional log files specified in the calling rule. If no log was specified, then an empty string “” is returned, regardless of the values of stdout, stderr, and append.

Parameters:

• stdout (bool) – Send stdout to log
• stderr (bool) – Send stderr to log
• append (bool) – Do not overwrite the log file. Useful for sending output of multiple commands to the same log. Note however that the log will not be truncated at the start.
• following table describes the output (The) –
• -------- -------- ----- ------------- (--------) –
• stderr append log return value (stdout) –
• -------- -------- ----- ------------ (--------) –
• True True fn >> fn 2>&1 (True) –
• False True fn >> fn (True) –
• True True fn 2>> fn (False) –
• True False fn > fn 2>&1 (True) –
• False False fn > fn (True) –
• True False fn 2> fn (False) –
• any any None "" (any) –
• -------- -------- ----- ----------- (--------) –

snakemake.script.get_source(path, basedir='.')

snakemake.script.script(path, basedir, input, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, jobid, bench_iteration, cleanup_scripts, shadow_dir)

Load a script from the given basedir + path and execute it.

snakemake.shell module

class snakemake.shell.shell

Bases: object

classmethod check_output(cmd, **kwargs)

classmethod cleanup()

classmethod executable(cmd)

classmethod get_executable()

static iter_stdout(proc, cmd)

classmethod kill(jobid)

classmethod prefix(prefix)

classmethod suffix(suffix)

snakemake.singularity module

snakemake.stats module

class snakemake.stats.Stats

Bases: object

file_stats

overall_runtime

report_job_end(job)

report_job_start(job)

rule_stats

to_json(path)

snakemake.utils module

class snakemake.utils.AlwaysQuotedFormatter(quote_func=None, *args, **kwargs)

Bases: snakemake.utils.QuotedFormatter

Subclass of QuotedFormatter that always quotes.

Usage is identical to QuotedFormatter, except that it always acts like “q” was appended to the format spec.

format_field(value, format_spec)

class snakemake.utils.QuotedFormatter(quote_func=None, *args, **kwargs)

Bases: string.Formatter

Subclass of string.Formatter that supports quoting.

Using this formatter, any field can be quoted after formatting by appending “q” to its format string. By default, shell quoting is performed using “shlex.quote”, but you can pass a different quote_func to the constructor. The quote_func simply has to take a string argument and return a new string representing the quoted form of the input string.

Note that if an element after formatting is the empty string, it will not be quoted.

format_field(value, format_spec)

snakemake.utils.R(code)

Execute R code.

This is deprecated in favor of the script directive. This function executes the R code given as a string. The function requires rpy2 to be installed.

Parameters:code (str) – R code to be executed

class snakemake.utils.SequenceFormatter(separator=' ', element_formatter=<string.Formatter object>, *args, **kwargs)

Bases: string.Formatter

string.Formatter subclass with special behavior for sequences.

This class delegates formatting of individual elements to another formatter object. Non-list objects are formatted by calling the delegate formatter’s “format_field” method. List-like objects (list, tuple, set, frozenset) are formatted by formatting each element of the list according to the specified format spec using the delegate formatter and then joining the resulting strings with a separator (space by default).

format_element(elem, format_spec)

Format a single element

For sequences, this is called once for each element in a sequence. For anything else, it is called on the entire object. It is intended to be overridden in subclases.

format_field(value, format_spec)

class snakemake.utils.Unformattable(errormsg='This cannot be used for formatting')

Bases: object

snakemake.utils.argvquote(arg, force=True)

Returns an argument quoted in such a way that that CommandLineToArgvW on Windows will return the argument string unchanged. This is the same thing Popen does when supplied with an list of arguments. Arguments in a command line should be separated by spaces; this function does not add these spaces. This implementation follows the suggestions outlined here: https://blogs.msdn.microsoft.com/twistylittlepassagesallalike/2011/04/23/everyone-quotes-command-line-arguments-the-wrong-way/

snakemake.utils.available_cpu_count()

Return the number of available virtual or physical CPUs on this system. The number of available CPUs can be smaller than the total number of CPUs when the cpuset(7) mechanism is in use, as is the case on some cluster systems.

Adapted from https://stackoverflow.com/a/1006301/715090

snakemake.utils.format(_pattern, *args, stepout=1, _quote_all=False, **kwargs)

Format a pattern in Snakemake style.

This means that keywords embedded in braces are replaced by any variable values that are available in the current namespace.

snakemake.utils.linecount(filename)

Return the number of lines of given file.

Parameters:filename (str) – the path to the file

snakemake.utils.listfiles(pattern, restriction=None, omit_value=None)

Yield a tuple of existing filepaths for the given pattern.

Wildcard values are yielded as the second tuple item.

Parameters:

• pattern (str) – a filepattern. Wildcards are specified in snakemake syntax, e.g. “{id}.txt”
• restriction (dict) – restrict to wildcard values given in this dictionary
• omit_value (str) – wildcard value to omit

Yields:

tuple – The next file matching the pattern, and the corresponding wildcards object

snakemake.utils.makedirs(dirnames)

Recursively create the given directory or directories without reporting errors if they are present.

snakemake.utils.min_version(version)

Require minimum snakemake version, raise workflow error if not met.

snakemake.utils.read_job_properties(jobscript, prefix='# properties', pattern=re.compile('# properties = (.*)'))

Read the job properties defined in a snakemake jobscript.

This function is a helper for writing custom wrappers for the snakemake –cluster functionality. Applying this function to a jobscript will return a dict containing information about the job.

snakemake.utils.report(text, path, stylesheet=None, defaultenc='utf8', template=None, metadata=None, **files)

Create an HTML report using python docutils.

This is deprecated in favor of the –report flag.

Attention: This function needs Python docutils to be installed for the python installation you use with Snakemake.

All keywords not listed below are intepreted as paths to files that shall be embedded into the document. They keywords will be available as link targets in the text. E.g. append a file as keyword arg via F1=input[0] and put a download link in the text like this:

report('''
==============
Report for ...
==============

Some text. A link to an embedded file: F1_.

Further text.
''', outputpath, F1=input[0])

Instead of specifying each file as a keyword arg, you can also expand
the input of your rule if it is completely named, e.g.:

report('''
Some text...
''', outputpath, **input)

Parameters:

• text (str) – The “restructured text” as it is expected by python docutils.
• path (str) – The path to the desired output file
• stylesheet (str) – An optional path to a css file that defines the style of the document. This defaults to <your snakemake install>/report.css. Use the default to get a hint how to create your own.
• defaultenc (str) – The encoding that is reported to the browser for embedded text files, defaults to utf8.
• template (str) – An optional path to a docutils HTML template.
• metadata (str) – E.g. an optional author name or email address.

snakemake.utils.simplify_path(path)

Return a simplified version of the given path.

snakemake.utils.update_config(config, overwrite_config)

Recursively update dictionary config with overwrite_config.

See https://stackoverflow.com/questions/3232943/update-value-of-a-nested-dictionary-of-varying-depth for details.

Parameters:

• config (dict) – dictionary to update
• overwrite_config (dict) – dictionary whose items will overwrite those in config

snakemake.utils.validate(data, schema, set_default=True)

Validate data with JSON schema at given path.

Parameters:

• data (object) – data to validate. Can be a config dict or a pandas data frame.
• schema (str) – Path to JSON schema used for validation. The schema can also be in YAML format. If validating a pandas data frame, the schema has to describe a row record (i.e., a dict with column names as keys pointing to row values). See https://json-schema.org. The path is interpreted relative to the Snakefile when this function is called.
• set_default (bool) – set default values defined in schema. See https://python-jsonschema.readthedocs.io/en/latest/faq/ for more information

snakemake.workflow module

class snakemake.workflow.RuleInfo(func)

Bases: object

class snakemake.workflow.Rules

Bases: object

A namespace for rules so that they can be accessed via dot notation.

class snakemake.workflow.Subworkflow(workflow, name, snakefile, workdir, configfile)

Bases: object

snakefile

target(paths)

targets(dag)

workdir

class snakemake.workflow.Workflow(snakefile=None, jobscript=None, overwrite_shellcmd=None, overwrite_config={}, overwrite_workdir=None, overwrite_configfiles=None, overwrite_clusterconfig={}, overwrite_threads={}, config_args=None, debug=False, verbose=False, use_conda=False, conda_prefix=None, use_singularity=False, use_env_modules=False, singularity_prefix=None, singularity_args='', shadow_prefix=None, mode=0, wrapper_prefix=None, printshellcmds=False, restart_times=None, attempt=1, default_remote_provider=None, default_remote_prefix='', run_local=True, default_resources=None, cache=None, nodes=1, cores=1, resources=None, conda_cleanup_pkgs=None)

Bases: object

add_rule(name=None, lineno=None, snakefile=None, checkpoint=False)

Add a rule.

apply_default_remote(path)

Apply the defined default remote provider to the given path and return the updated _IOFile. Asserts that default remote provider is defined.

benchmark(benchmark)

cache_rule(cache)

check()

check_localrules()

concrete_files

conda(conda_env)

config

configfile(fp)

Update the global config with data from the given file.

container(container_img)

cores

current_basedir

Basedir of currently parsed Snakefile.

cwl(cwl)

docstring(string)

envmodules(*env_modules)

execute(targets=None, dryrun=False, touch=False, local_cores=1, forcetargets=False, forceall=False, forcerun=None, until=[], omit_from=[], prioritytargets=None, quiet=False, keepgoing=False, printshellcmds=False, printreason=False, printdag=False, cluster=None, cluster_sync=None, jobname=None, immediate_submit=False, ignore_ambiguity=False, printrulegraph=False, printfilegraph=False, printd3dag=False, drmaa=None, drmaa_log_dir=None, kubernetes=None, tibanna=None, tibanna_sfn=None, precommand='', tibanna_config=False, container_image=None, stats=None, force_incomplete=False, ignore_incomplete=False, list_version_changes=False, list_code_changes=False, list_input_changes=False, list_params_changes=False, list_untracked=False, list_conda_envs=False, summary=False, archive=None, delete_all_output=False, delete_temp_output=False, detailed_summary=False, latency_wait=3, wait_for_files=None, nolock=False, unlock=False, notemp=False, nodeps=False, cleanup_metadata=None, conda_cleanup_envs=False, cleanup_shadow=False, cleanup_scripts=True, subsnakemake=None, updated_files=None, keep_target_files=False, keep_shadow=False, keep_remote_local=False, allowed_rules=None, max_jobs_per_second=None, max_status_checks_per_second=None, greediness=1.0, no_hooks=False, force_use_threads=False, conda_create_envs_only=False, assume_shared_fs=True, cluster_status=None, report=None, report_stylesheet=None, export_cwl=False, batch=None, keepincomplete=False)

get_rule(name)

Get rule by name.

Arguments name – the name of the rule

get_sources()

global_container(container_img)

global_wildcard_constraints(**content)

Register global wildcard constraints.

group(group)

include(snakefile, overwrite_first_rule=False, print_compilation=False, overwrite_shellcmd=None)

Include a snakefile.

input(*paths, **kwpaths)

inputfile(path)

Mark file as being an input file of the workflow.

This also means that eventual –default-remote-provider/prefix settings will be applied to this file. The file is returned as _IOFile object, such that it can e.g. be transparently opened with _IOFile.open().

is_cached_rule(rule: snakemake.rules.Rule)

is_local(rule)

is_rule(name)

Return True if name is the name of a rule.

Arguments name – a name

lint(json=False)

list_resources()

list_rules(only_targets=False)

localrules(*rulenames)

log(*logs, **kwlogs)

message(message)

nodes

norun()

notebook(notebook)

onerror(func)

Register onerror function.

onstart(func)

Register onstart function.

onsuccess(func)

Register onsuccess function.

output(*paths, **kwpaths)

params(*params, **kwparams)

priority(priority)

register_envvars(*envvars)

Register environment variables that shall be passed to jobs. If used multiple times, union is taken.

report(path)

Define a global report description in .rst format.

resources(*args, **resources)

rule(name=None, lineno=None, snakefile=None, checkpoint=False)

ruleorder(*rulenames)

rules

run(func)

script(script)

shadow(shadow_depth)

shellcmd(cmd)

subworkflow(name, snakefile=None, workdir=None, configfile=None)

subworkflows

threads(threads)

version(version)

wildcard_constraints(*wildcard_constraints, **kwwildcard_constraints)

workdir(workdir)

Register workdir.

wrapper(wrapper)

snakemake.workflow.format_resources(dict_like, *, omit_keys={'_cores', '_nodes'}, omit_values=[])

snakemake.workflow.srcdir(path)

Return the absolute path, relative to the source directory of the current Snakefile.

snakemake.wrapper module

snakemake.wrapper.find_extension(path, extensions=['.py', '.R', '.Rmd', '.jl'])

snakemake.wrapper.get_conda_env(path, prefix=None)

snakemake.wrapper.get_path(path, prefix=None)

snakemake.wrapper.get_script(path, prefix=None)

snakemake.wrapper.is_git_path(path)

snakemake.wrapper.is_local(path)

snakemake.wrapper.is_script(path)

snakemake.wrapper.wrapper(path, input, output, params, wildcards, threads, resources, log, config, rulename, conda_env, container_img, singularity_args, env_modules, bench_record, prefix, jobid, bench_iteration, cleanup_scripts, shadow_dir)

Load a wrapper from https://github.com/snakemake/snakemake-wrappers under the given path + wrapper.(py|R|Rmd) and execute it.

Module contents

snakemake.bash_completion(snakefile='Snakefile')

Entry point for bash completion.

snakemake.get_appdirs()

snakemake.get_argument_parser(profile=None)

Generate and return argument parser.

snakemake.get_profile_file(profile, file, return_default=False)

snakemake.main(argv=None)

Main entry point.

snakemake.parse_batch(args)

snakemake.parse_config(args)

Parse config from args.

snakemake.parse_key_value_arg(arg, errmsg)

snakemake.parse_set_threads(args)

snakemake.snakemake(snakefile, batch=None, cache=None, report=None, report_stylesheet=None, lint=None, listrules=False, list_target_rules=False, cores=1, nodes=1, local_cores=1, resources={}, overwrite_threads={}, default_resources=None, config={}, configfiles=None, config_args=None, workdir=None, targets=None, dryrun=False, touch=False, forcetargets=False, forceall=False, forcerun=[], until=[], omit_from=[], prioritytargets=[], stats=None, printreason=False, printshellcmds=False, debug_dag=False, printdag=False, printrulegraph=False, printfilegraph=False, printd3dag=False, nocolor=False, quiet=False, keepgoing=False, cluster=None, cluster_config=None, cluster_sync=None, drmaa=None, drmaa_log_dir=None, jobname='snakejob.{rulename}.{jobid}.sh', immediate_submit=False, standalone=False, ignore_ambiguity=False, snakemakepath=None, lock=True, unlock=False, cleanup_metadata=None, conda_cleanup_envs=False, cleanup_shadow=False, cleanup_scripts=True, force_incomplete=False, ignore_incomplete=False, list_version_changes=False, list_code_changes=False, list_input_changes=False, list_params_changes=False, list_untracked=False, list_resources=False, summary=False, archive=None, delete_all_output=False, delete_temp_output=False, detailed_summary=False, latency_wait=3, wait_for_files=None, print_compilation=False, debug=False, notemp=False, keep_remote_local=False, nodeps=False, keep_target_files=False, allowed_rules=None, jobscript=None, greediness=None, no_hooks=False, overwrite_shellcmd=None, updated_files=None, log_handler=[], keep_logger=False, max_jobs_per_second=None, max_status_checks_per_second=100, restart_times=0, attempt=1, verbose=False, force_use_threads=False, use_conda=False, use_singularity=False, use_env_modules=False, singularity_args='', conda_prefix=None, conda_cleanup_pkgs=None, list_conda_envs=False, singularity_prefix=None, shadow_prefix=None, conda_create_envs_only=False, mode=0, wrapper_prefix=None, kubernetes=None, container_image=None, tibanna=False, tibanna_sfn=None, precommand='', default_remote_provider=None, default_remote_prefix='', tibanna_config=False, assume_shared_fs=True, cluster_status=None, export_cwl=None, show_failed_logs=False, keep_incomplete=False, messaging=None)

Run snakemake on a given snakefile.

This function provides access to the whole snakemake functionality. It is not thread-safe.

Parameters:

• snakefile (str) – the path to the snakefile
• batch (Batch) – whether to compute only a partial DAG, defined by the given Batch object (default None)
• report (str) – create an HTML report for a previous run at the given path
• lint (str) – print lints instead of executing (None, “plain” or “json”, default None)
• listrules (bool) – list rules (default False)
• list_target_rules (bool) – list target rules (default False)
• cores (int) – the number of provided cores (ignored when using cluster support) (default 1)
• nodes (int) – the number of provided cluster nodes (ignored without cluster support) (default 1)
• local_cores (int) – the number of provided local cores if in cluster mode (ignored without cluster support) (default 1)
• resources (dict) – provided resources, a dictionary assigning integers to resource names, e.g. {gpu=1, io=5} (default {})
• default_resources (DefaultResources) – default values for resources not defined in rules (default None)
• config (dict) – override values for workflow config
• workdir (str) – path to working directory (default None)
• targets (list) – list of targets, e.g. rule or file names (default None)
• dryrun (bool) – only dry-run the workflow (default False)
• touch (bool) – only touch all output files if present (default False)
• forcetargets (bool) – force given targets to be re-created (default False)
• forceall (bool) – force all output files to be re-created (default False)
• forcerun (list) – list of files and rules that shall be re-created/re-executed (default [])
• prioritytargets (list) – list of targets that shall be run with maximum priority (default [])
• stats (str) – path to file that shall contain stats about the workflow execution (default None)
• printreason (bool) – print the reason for the execution of each job (default false)
• printshellcmds (bool) – print the shell command of each job (default False)
• printdag (bool) – print the dag in the graphviz dot language (default False)
• printrulegraph (bool) – print the graph of rules in the graphviz dot language (default False)
• printfilegraph (bool) – print the graph of rules with their input and output files in the graphviz dot language (default False)
• printd3dag (bool) – print a D3.js compatible JSON representation of the DAG (default False)
• nocolor (bool) – do not print colored output (default False)
• quiet (bool) – do not print any default job information (default False)
• keepgoing (bool) – keep goind upon errors (default False)
• cluster (str) – submission command of a cluster or batch system to use, e.g. qsub (default None)
• cluster_config (str,list) – configuration file for cluster options, or list thereof (default None)
• cluster_sync (str) – blocking cluster submission command (like SGE ‘qsub -sync y’) (default None)
• drmaa (str) – if not None use DRMAA for cluster support, str specifies native args passed to the cluster when submitting a job
• drmaa_log_dir (str) – the path to stdout and stderr output of DRMAA jobs (default None)
• jobname (str) – naming scheme for cluster job scripts (default “snakejob.{rulename}.{jobid}.sh”)
• immediate_submit (bool) – immediately submit all cluster jobs, regardless of dependencies (default False)
• standalone (bool) – kill all processes very rudely in case of failure (do not use this if you use this API) (default False) (deprecated)
• ignore_ambiguity (bool) – ignore ambiguous rules and always take the first possible one (default False)
• snakemakepath (str) – deprecated parameter whose value is ignored. Do not use.
• lock (bool) – lock the working directory when executing the workflow (default True)
• unlock (bool) – just unlock the working directory (default False)
• cleanup_metadata (list) – just cleanup metadata of given list of output files (default None)
• conda_cleanup_envs (bool) – just cleanup unused conda environments (default False)
• cleanup_shadow (bool) – just cleanup old shadow directories (default False)
• cleanup_scripts (bool) – delete wrapper scripts used for execution (default True)
• force_incomplete (bool) – force the re-creation of incomplete files (default False)
• ignore_incomplete (bool) – ignore incomplete files (default False)
• list_version_changes (bool) – list output files with changed rule version (default False)
• list_code_changes (bool) – list output files with changed rule code (default False)
• list_input_changes (bool) – list output files with changed input files (default False)
• list_params_changes (bool) – list output files with changed params (default False)
• list_untracked (bool) – list files in the workdir that are not used in the workflow (default False)
• summary (bool) – list summary of all output files and their status (default False)
• archive (str) – archive workflow into the given tarball
• delete_all_output (bool) remove all files generated by the workflow (default False) –
• delete_temp_output (bool) remove all temporary files generated by the workflow (default False) –
• latency_wait (int) – how many seconds to wait for an output file to appear after the execution of a job, e.g. to handle filesystem latency (default 3)
• wait_for_files (list) – wait for given files to be present before executing the workflow
• list_resources (bool) – list resources used in the workflow (default False)
• summary – list summary of all output files and their status (default False). If no option is specified a basic summary will be ouput. If ‘detailed’ is added as an option e.g –summary detailed, extra info about the input and shell commands will be included
• detailed_summary (bool) – list summary of all input and output files and their status (default False)
• print_compilation (bool) – print the compilation of the snakefile (default False)
• debug (bool) – allow to use the debugger within rules
• notemp (bool) – ignore temp file flags, e.g. do not delete output files marked as temp after use (default False)
• keep_remote_local (bool) – keep local copies of remote files (default False)
• nodeps (bool) – ignore dependencies (default False)
• keep_target_files (bool) – do not adjust the paths of given target files relative to the working directory.
• allowed_rules (set) – restrict allowed rules to the given set. If None or empty, all rules are used.
• jobscript (str) – path to a custom shell script template for cluster jobs (default None)
• greediness (float) – set the greediness of scheduling. This value between 0 and 1 determines how careful jobs are selected for execution. The default value (0.5 if prioritytargets are used, 1.0 else) provides the best speed and still acceptable scheduling quality.
• overwrite_shellcmd (str) – a shell command that shall be executed instead of those given in the workflow. This is for debugging purposes only.
• updated_files (list) – a list that will be filled with the files that are updated or created during the workflow execution
• verbose (bool) – show additional debug output (default False)
• max_jobs_per_second (int) – maximal number of cluster/drmaa jobs per second, None to impose no limit (default None)
• restart_times (int) – number of times to restart failing jobs (default 0)
• attempt (int) – initial value of Job.attempt. This is intended for internal use only (default 1).
• force_use_threads – whether to force use of threads over processes. helpful if shared memory is full or unavailable (default False)
• use_conda (bool) – use conda environments for each job (defined with conda directive of rules)
• use_singularity (bool) – run jobs in singularity containers (if defined with singularity directive)
• use_env_modules (bool) – load environment modules if defined in rules
• singularity_args (str) – additional arguments to pass to singularity
• conda_prefix (str) – the directory in which conda environments will be created (default None)
• conda_cleanup_pkgs (snakemake.deployment.conda.CondaCleanupMode) – whether to clean up conda tarballs after env creation (default None), valid values: “tarballs”, “cache”
• singularity_prefix (str) – the directory to which singularity images will be pulled (default None)
• shadow_prefix (str) – prefix for shadow directories. The job-specific shadow directories will be created in $SHADOW_PREFIX/shadow/ (default None)
• conda_create_envs_only (bool) – if specified, only builds the conda environments specified for each job, then exits.
• list_conda_envs (bool) – list conda environments and their location on disk.
• mode (snakemake.common.Mode) – execution mode
• wrapper_prefix (str) – prefix for wrapper script URLs (default None)
• kubernetes (str) – submit jobs to kubernetes, using the given namespace.
• container_image (str) – Docker image to use, e.g., for kubernetes.
• default_remote_provider (str) – default remote provider to use instead of local files (e.g. S3, GS)
• default_remote_prefix (str) – prefix for default remote provider (e.g. name of the bucket).
• tibanna (str) – submit jobs to AWS cloud using Tibanna.
• tibanna_sfn (str) – Step function (Unicorn) name of Tibanna (e.g. tibanna_unicorn_monty). This must be deployed first using tibanna cli.
• precommand (str) – commands to run on AWS cloud before the snakemake command (e.g. wget, git clone, unzip, etc). Use with –tibanna.
• tibanna_config (list) – Additional tibanan config e.g. –tibanna-config spot_instance=true subnet=<subnet_id> security group=<security_group_id>
• assume_shared_fs (bool) – assume that cluster nodes share a common filesystem (default true).
• cluster_status (str) – status command for cluster execution. If None, Snakemake will rely on flag files. Otherwise, it expects the command to return “success”, “failure” or “running” when executing with a cluster jobid as single argument.
• export_cwl (str) – Compile workflow to CWL and save to given file
• log_handler (list) – redirect snakemake output to this custom log handler, a function that takes a log message dictionary (see below) as its only argument (default None). The log message dictionary for the log handler has to following entries:
• keep_incomplete (bool) – keep incomplete output files of failed jobs
• log_handler –

  redirect snakemake output to this list of custom log handler, each a function that takes a log message dictionary (see below) as its only argument (default []). The log message dictionary for the log handler has to following entries:

  level:the log level (“info”, “error”, “debug”, “progress”, “job_info”) level=”info”, “error” or “debug”:  

  msg:the log message

  level=”progress”:  

  done:number of already executed jobs total:number of total jobs

  level=”job_info”:  

  input:list of input files of a job output:list of output files of a job log:path to log file of a job local:whether a job is executed locally (i.e. ignoring cluster) msg:the job message reason:the job reason priority:the job priority threads:the threads of the job

Returns:

True if workflow execution was successful.

Return type:

bool

snakemake.unparse_config(config)