Snakemake: Running hybrid workflows

user_18092798 0
Viewed 47

I was wondering if there is any way to configure snakemake so that certain rules can be run locally while others are run on an cluster.

For example, suppose I had rule A, which is lightweight, and rule B, which is more computationally expensive. Rule B requires the output of rule A. Is there any way for me to run rule A on a local computer and rule B be submitted as a job via SLURM?

EDIT: I should have read the documentation more thoroughly. What I said above can be accomplished using localrules.

As a modification to the above question, can I run one rule on a cluster and one rule on the cloud?

snakemake bioinformatics python
1 Answers

A probably cluncky solution is to use ssh to execute commands on the remote host. I don't use the cloud but I assume you can ssh there or do something equivalent. So something like:

rule one:
    input:
        'data.txt',
    output:
        'results.txt',
    shell:
        r"""
        rsync -arvP {input} me@123.123.123:~/
        ssh me@123.123.123 "sort {input} > {output}"
        rsync -arvP me@123.123.123:~/{output} ./ 
        """
user_11144530
Related