Arithmetic on "threads" wildcard in rule definition

user_459780 0
Viewed 252

In my workflow, I have two multithreaded programs that I am piping together. See the rule definition from my Snakemake file below.

rule do_the_thing:
    input: 'input.dat'
    output: 'output.dat'
    threads: 16
    shell: 'cmd1 --threads {threads} {input} | cmd2 --threads {threads} > {output}'

As written, the command will consume 32 threads. Is there a way I can perform arithmetic operations on the threads variable, so that (for instance) each command consumes only half of the available threads?

snakemake multithreading python
1 Answers

There are quite some options:

shell: 
    cpulimit --include-children -l {threads}00 --\
    cmd1 --threads {threads} {input} | cmd2 --threads {threads} > {output}
  • calculate the thread usage in the params:
threads: 16
params: lambda threads: max(1, threads//2)
shell: 
    cmd1 --threads {params.threads} {input} | cmd2 --threads {params.threads} > {output}

rule do_the_thing_first:
    output: pipe('output.pipe')
    threads: 8
    shell: 'cmd1 --threads {threads} {input} > {output}'

rule do_the_thing_second:
    input: 'output.pipe'
    output: 'output.dat'
    threads: 8
    shell: 'cmd2 --threads {threads} > {output}'
user_95445160
Related