For Loop with Filename Pairing

user_11689123 0
Viewed 63

I am trying to create a script that will, within a for loop, identify pairs of files in a directory and then perform a function on each pair. The paired files are named such as FILENAME_1.fastq and FILENAME_2.fastq and there are multiple pairs within the directory. Here are some actual filenames in case it matters for regex functions:

WT1_0min-SRR9929263_1.fastq
WT1_0min-SRR9929263_2.fastq
WT1_20min-SRR9929265_1.fastq
WT1_20min-SRR9929265_2.fastq
WT3_20min-SRR12062597_1.fastq
WT3_20min-SRR12062597_2.fastq

Is it possible to do this without feeding any information about the filename, other than that it has a pair? I am absolutely awful with regex and name-search functions, but below is my latest failed attempt.

cd ~/Directory

for file in *.fastq
do
  sample=`basename ${file}` #I think needs a modification to subtract the _1 or _2 and then a search function to find the paired files
  myfunction \
         -1 ${sample}_1.fastq \
         -2 ${sample}_2.fastq \
done

Thanks for any help. Been stuck for 2 days x_x

for-loop awk
3 Answers

to account for the scenario not every file is fully paired , try

file . -depth 1 type f -not -name ".*" | \
 \
  gawk 'BEGIN { FS="_"; } { $ 0 = gensub(/^.+\/([^\/]+)$/ , "\\1", "1"); } 
              { inL[$1$2][substr($3,1,1)] = $0 ; }   
          END { OFS = ORS = "\0"; 
                    for (pfx in inL) { 
                        if (1 in inL[pfx]) && (2 in inL[pfx]) && \
                           (length(inL[pfx])==2)
              { print inL[pfx][1], inL[pfx][2]; } } }' | \
  \ 
  parallel -0 -N 2 -j 1 myfunction -1 '{}' -2 '{}' ;

gnu parallel allows for functions to be exported. this version of the code will use gawk to handle both basename and print0 functions. This also ensures ONLY files with exact 1+2 parings will be shown in the end, in case there are ones with only one of the 2, or some files with even a "_3.fastq", shall your need to expand into such a realm.

user_146721140

Use find and xargs, and replace echo with the command of your choice:

find . -name '*_1.fastq' -exec basename {} '_1.fastq' \; | xargs -n1 -I{} echo {}_1.fastq {}_2.fastq
user_9676210

In bioinformatics we use this all the time when having a paired end run:

parallel --plus echo {} {/_1.fastq/_2.fastq} ::: *_1.fastq

or:

parallel echo {} {=s/_1.fastq/_2.fastq/=} ::: *_1.fastq
user_3630280
Related