How to download a list of `FastQ` files in `Nextflow` using `fromSRA` function?

user_2572287 0
Viewed 222

I have a tsv file with various columns. One of the columns of interest for me is the run_accession column. It contains accession id of various genome data samples. I want to write a pipeline in Nextflow which reads accession ids from this file using the following command:

cut -f4 datalist.tsv | sed -n 2,11p

Output:

ERR2512385  
ERR2512386  
ERR2512387  
ERR2512388  
ERR2512389  
ERR2512390  
ERR2512391  
ERR2512392  
ERR2512393  
ERR2512394

and feed this list of IDs into Channel.fromSRA method. So far, I have tried this:

#!/home/someuser/bin nextflow

nextflow.enable.dsl=2

params.datalist = "$baseDir/datalist.tsv"

process fetchRunAccession {
    input:
    path dlist

    output:
    file accessions

    """
    cut -f4 $dlist | sed -n 2,11p
    """
}

process displayResult {
    input:
    file accessions

    output:
    stdout

    """
    echo "$accessions"
    """
}

workflow {
    accessions_p = fetchRunAccession(params.datalist)
    result = displayResult(accessions_p)
    result.view { it }
}

And I get this error:

Error executing process > 'fetchRunAccession'

Caused by:
  Missing output file(s) `accessions` expected by process `fetchRunAccession

If I run just the first process it works well and prints 10 lines as expected. The second process is just a placeholder for the actual fromSRA implementation but I have not been able to use the output of first process as the input of second. I am very new to Nextflow and my code probably has some silly mistakes. I would appreciate any help in this matter.

nextflow bioinformatics
1 Answers

The fromSRA function is actually a factory method. It requires either a project/study id, or one or more accession numbers, which must be specified as a list. A channel emitting accession numbers (like in your example code) will not work here. Also, it would be better to avoid spawning a separate job/process just to parse a small CSV file. Instead, just let your main Nextflow process do this. There's lots of ways to do this, but for CSV input I find using Nextflow's CsvSplitter class makes this easy:

import nextflow.splitter.CsvSplitter

nextflow.enable.dsl=2


def fetchRunAccessions( tsv ) {

    def splitter = new CsvSplitter().options( header:true, sep:'\t' )
    def reader = new BufferedReader( new FileReader( tsv ) )

    splitter.parseHeader( reader )

    List<String> run_accessions = []
    Map<String,String> row

    while( row = splitter.fetchRecord( reader ) ) {

       run_accessions.add( row['run_accession'] )
    }

    return run_accessions
}


workflow {

    accessions = fetchRunAccessions( params.filereport )

    Channel
        .fromSRA( accessions )
        .view()
}

Note that Nextflow's ENA download URL was updated recently. You'll need the latest version of Nextflow (21.07.0-edge) to get this to run easily:

NXF_VER=21.07.0-edge nextflow run test.nf --filereport filereport.tsv
user_7518630
Related