snakemake allocates memory twice

Viewed 26

I am noticing that all my rules request memory twice, one at a lower maximum than what I requested (mem_mb) and then what I actually requested (mem_gb). If I run the rules as localrules they do run faster. How can I make sure the default settings do not interfere?

resources: mem_mb=100, disk_mb=8620, tmpdir=/tmp/pop071.54835, partition=h24, qos=normal, mem_gb=100, time=120:00:00

The rules are as follows:

rule bwa_mem2_mem:
    input:
        R1 = "data/results/qc/{species}.{population}.{individual}_1.fq.gz",
        R2 = "data/results/qc/{species}.{population}.{individual}_2.fq.gz", 
        R1_unp = "data/results/qc/{species}.{population}.{individual}_1_unp.fq.gz",
        R2_unp = "data/results/qc/{species}.{population}.{individual}_2_unp.fq.gz",
        idx= "data/results/genome/genome",
        ref = "data/results/genome/genome.fa"
    output:
        bam = "data/results/mapped_reads/{species}.{population}.{individual}.bam",
    log:
        bwa ="logs/bwa_mem2/{species}.{population}.{individual}.log",
        sam ="logs/samtools_view/{species}.{population}.{individual}.log",
    benchmark:
        "benchmark/bwa_mem2_mem/{species}.{population}.{individual}.tsv",
    resources:
        time = parameters["bwa_mem2"]["time"],
        mem_gb = parameters["bwa_mem2"]["mem_gb"],        
    params:
        extra = parameters["bwa_mem2"]["extra"],
        tag = compose_rg_tag,
    threads:
        parameters["bwa_mem2"]["threads"],
    shell:
        "bwa-mem2 mem -t {threads} -R '{params.tag}' {params.extra} {input.idx} {input.R1} {input.R2} | "
        "samtools sort -l 9 -o {output.bam} --reference {input.ref} --output-fmt CRAM -@ {threads} /dev/stdin 2> {log.sam}"

and the config is:

cluster:
  mkdir -p logs/{rule} && # change the log file to logs/slurm/{rule}
  sbatch
    --partition={resources.partition}
    --time={resources.time}
    --qos={resources.qos}
    --cpus-per-task={threads}
    --mem={resources.mem_gb}
    --job-name=smk-{rule}-{wildcards}
    --output=logs/{rule}/{rule}-{wildcards}-%j.out
    --parsable # Required to pass job IDs to scancel
default-resources:
  - partition=h24
  - qos=normal
  - mem_gb=100
  - time="04:00:00"
restart-times: 3
max-jobs-per-second: 10
max-status-checks-per-second: 1
local-cores: 1
latency-wait: 60
jobs: 100
keep-going: True
rerun-incomplete: True
printshellcmds: True
scheduler: greedy
use-conda: True # Required to run with local conda enviroment
cluster-status: status-sacct.sh # Required to monitor the status of the submitted jobs
cluster-cancel: scancel # Required to cancel the jobs with Ctrl + C 
cluster-cancel-nargs: 50

Cheers, Angel

1 Answers

Right now there are two separate memory resource requirements:

  • mem_mb
  • mem_gb

From the perspective of snakemake these are different, so both will be passed to the cluster. A quick fix is to use the same units, e.g. if the resource really requires only 100 mb, then the default resource should be changed to:

default-resources:
  - partition=h24
  - qos=normal
  - mem_mb=100
Related