Snakemake, numerous rules with different sets of wildcards

user_15994275 0
Viewed 29

I would like to write a series of rules, but some rules do not use the same wildcards as other rules. When I try this, Snakemake cannot determine the wildcards.

Suppose I have a workflow like this:

  1. generate separate data files for 30 chromosomes
  2. run a program called "algorithm1" with parameter "foo" on each of these 30 chromosomes
  3. run a different program called "algorithm2" with parameter "bar" on each of these 30 chromosomes
  4. Algorithm 1 and 2 are not related; I want algorithm 1 to run then afterwards algortihm2.

An illustrative workflow which you can run at home (changing user) looks like this:

rule all:
    input:
        [f'/home/user/snakemake_test_220905/algorithm2/parameter1_{alg2_param1}/littlefile{chrom}_analysis.txt' for chrom in range(0,10) for alg1_param1 in ['foo'] for alg2_param1 in ['bar']]

    
rule write_separate_vcf:
    input: 
        infile = '/home/user/snakemake_test_220905/big_file.vcf'
    output: 
        outfile = '/home/user/snakemake_test_220905/littlefile{chrom}.vcf'
    run:
        shell(
            f'echo "file{wildcards.chrom}" > {output.outfile}'
        )

rule algorithm_1:
    input: 
        infile = '/home/user/snakemake_test_220905/littlefile{chrom}.vcf'
    output:
        outfile = '/home/user/snakemake_test_220905/algorithm1/parameter1_{alg1_param1}/littlefile{chrom}_analysis.txt'
    run:
        shell(
            f'echo "parameter_{wildcards.alg1_param1}_{input.infile}" > {output.outfile}'
        )

rule algorithm_2:
    input: 
        infile = '/home/user/snakemake_test_220905/littlefile{chrom}.vcf',
        inile_flag = '/home/user/snakemake_test_220905/algorithm1/parameter1_{alg1_param1}/littlefile{chrom}_analysis.txt' # force dependency
    output:
        outfile = '/home/user/snakemake_test_220905/algorithm2/parameter1_{alg2_param1}/littlefile{chrom}_analysis.txt'
    run:
        shell(
            f'echo "parameter_{wildcards.alg2_param1}_{input.infile}" > {output.outfile}'
        )

(use for i in {1..10}; do echo vcf$i >> big_file.vcf; done to create the big_file.vcf)

This of course fails, the returned error is:

WildcardError in line 29 of /home/user/snakemake_test_220905/Snakefile:
Wildcards in input files cannot be determined from output files:
'alg1_param1'

I would like all the files to be written at once, but can't figure out an elegant way to do it. Any ideas?

snakemake python
1 Answers

The output of rule algorithm_2 should contain {alg1_param1} otherwise different values of {alg1_param1} would lead to the same output. This is not a limitation of snakemake, if it was allowed you would have files overwriting each other. For example use:

outfile = 'algorithm2/{alg1_param1}/parameter1_{alg2_param1}/littlefile{chrom}_analysis.txt'

Output of rule all should change accordingly.

user_11144530
Related