How to generate multiple output files from single input file that does not share naming convention in snakemake?

user_16487557 0
Viewed 161

I have searched this for some time now, and this thread is the closest I got, but could not get working with my setup.

What I want to do:

I have one text file where every line has an ID and a data point

1234 data2
5678 data3
...

I want to collect the lines that correspond to certain IDs, which I have in my config file, and write them to their own files named according to the IDs value (1234 or 5678)

# config.yaml
IDs:
    ID1: 1234
    ID2: 5678

When I did this without snakemake, I just looped over the list of IDs in my bash script and grepped the text file for them, but I just cannot accomplish this with snakemake.

Either I have an issue with wildcards in target, or my expand function gives all of the IDs to the grep command in shell, or when following that accepted linked answer, I get "missing input files for rule all: And_Laa A_log" I can share what I have now, but I think the correct way to do this is so far removed from what I have, that it will just confuse everyone:

configfile: "config.yaml"

# Trying to replicate stackoverflow answer
speakers = {
  "1": "And_Laa",
  "2": "A_log"
}

def get_speaker(wildcards):
#  return expand("{speaker}", speaker=config["speakers"]) 
  return speakers[wildcards.speaker]

rule all:
  input:
#    expand("{speaker}_wav-list", speaker=config[speakers])
    expand("{speaker}", speaker=speakers.values())

# Selecting all the audiofiles for the speakers from a very large file
rule select_speaker_files:
  input:
    wav=config["files"]["wavs"]
  output:
    speaker="{speaker}_wav-list"
  params:
    speaker=get_speaker,
  shell:
    'grep "{params.speaker}" {input.wav} > {output.speaker}'
snakemake
1 Answers

First of all, I guess that what you call a "speaker" is not a value of the dict, but it's key. So you rule all should expand like that:

rule all:
  input:
    expand("{speaker}", speaker=speakers)

Next, this rule literally says: "I require two files with the filenames 1 and 2." But there is no rule that produce the files with these names. You have:

rule select_speaker_files:
  output:
    speaker="{speaker}_wav-list"

Thir rule claims: "I can produce a file which name ends in _wav-list." Definitely there are no rules that may produce something that the pipeline needs to create. You probably meant that:

rule all:
  input:
    expand("{speaker}_wav-list", speaker=speakers)

In this case the rules at least are in consistency.

user_112186870
Related