How to echo variables within awk to create a new file

user_4701887 0
Viewed 62

I have a test_1.fastq file below and the code to split the file by matching the header. In addition to this, I want to add another line of code like echo "@RG\tID:"arr[1]"\tPL:illumina\tPU:${SM}" > ${SM}."arr[1]"."lane".rgfile" . Basically, I want to create a new file by printing the echo statement appending the content of "arr[1]"."lane" in the code below. So, In addition to what my code produces, I also want to create three additional files ${SM}."arr[1]"."lane".rgfile". What do I need to do for this?

Code I have:

FQ_OUT1="test_1.fastq"
SM="sample1"
awk -v sm="$SM" '
BEGIN{FS = ":"} 
/^@HS2000/{
  split($1,arr,"_")
  sub(/^@+/,"",arr[1])
  lane=$2
  close(outputFile)  
  outputFile=sm"."arr[1]"."lane".fastq"
}
{
  print >> (outputFile)
}' ${FQ_OUT1}

test_1.fastq:

@HS2000-1015_160:7:1108:13370:100570/2
CTTGACTGCCAGAGACGCTCCTTTGCAATGCCTTCCGGTAACCAAATTTTTGGGCACAACACACAGCTGGCCTTCATTTCTTCAGGGGCTGGTAAACAGA
+
@@@ADFFFHHHFD=EF@:GHIIFHH<ECHGF@DDBB:6@D60?F=888)8='--(=5@EAE5?'(..((.;?@>>A>3;@####################
@HS2000-1015_160:5:2306:10070:71746/2
GAACCTCAAGGACTATTGGGAGAGCGGCGAGTGGGCCATCATCAAAGCCCCAGGCTACAAACACGACATCAAGTACAACTGCTGCGAGGAGATCTACCCC
+
@CCFFFFDHGHFHIJJJJJJGGGIIJJIGHI@FHGIIGHHEFGHHFFFFFBCDDDDDCDDDDDDDD;@BDCCDACDD@>ACCDDDDBDB<BA?C@CC@BD
@HS2000-1015_160:6:2116:4077:79041/2
GGTCCCCGCCTACGCCCACTGGGTTGGTGCACCTGGTGGTGGTGGCCGCCAAGAAGCTGGTGAACCGCCTCCAAGTGGCTCCCAAGACGCAGCTGGATGA
+
CCCFFFFFHHFHHJHJJJJJJJJGGHJHIGAGIIJIFHJ;@F;CHHFHFDDDDDCDDCDD9CCCDDBDDBBDDCDACDD8@BD3>?BCDBDDDACCDC@>
@HS2000-1015_160:5:2113:11446:94436/2
CGTCAGGGCCAACCCCGCCCCACCCTGACCCTACCTGGCACCCCTCACCTGTGGCCTGCCAGCACAGCCTCGCCCCTGCTGGCCAATGTGTCCCCCGTCA
+
?@@DA@DDFHH?DHI)<@@FHDBGGCHCBDH;DFA<)6.=7D;@CBCHD)).7@=>;?==AABC95<(5(5309@D########################
@HS2000-1015_160:6:2209:18284:44195/2
TAAAATGTCACAAAGCTGGAAACTCTTCCCTATCACAAACCAAAACTTAAAAGGACGTTACCTGGCTGGGTCTAAACTCCACATAACTCGCTTGCAGTTG
+
CCCFFFFEHHHGHJIIIJJIJJHIIJEHJJHIJJJIIJJIJIJJIJIIHJJIJGGHGHGIIHHIIIIHFH@DFFFDEEEECDDDCDDDDBDDBBDCDACC
@HS2000-1015_160:7:1215:18781:100685/2
ATAAAACAGTAAACAAAATAAAGTCAGTTTTTTTTTTTTTTTTTAAAGAACAAAATGAAACTTGAGGGAAAACTTCATGGAGTTACAGTTTATCCTGATA
+
CCCFFFFFHFHHFJJJJIIGIGI<CFHHIIJJJJJIJJHFDDD=ACC(38+9CB?:(>C(+:@>(4?05<?C?###########################
@HS2000-1015_160:6:1215:6292:43622/2
GGGTCCTGAGACCTGAGGGACCATTGGCCCTCTTCTGGCTTGCTTATCCTTTGTACCTGATGGCCAATGAATGTCAGAGATGGTCCTGTCTCCATCCAGT
+
BCCDFFFFHGHHHJJJIIJJJJJIJJJGIJIJJIHIIJJIEFHEIJJJJIGIGIIIIIJHFHIJJJJIHGHEC?BCEFFFEECCCEACCCCDDDDDDCCC
awk shell linux
1 Answers

Looks like you want to print only 1 line into new output file, considering that line should be added one time and when output file name is getting created on limes which are starting with @HS, if this is the case then try following. As per OP's comments OP wants to print \t ad a literal hence edited solution as per that here.

FQ_OUT1="test_1.fastq"
SM="sample1"
awk -v sm="$SM" -v tab='\\t' '
BEGIN{FS = ":"} 
/^@HS2000/{
  split($1,arr,"_")
  sub(/^@+/,"",arr[1])
  lane=$2
  close(outputFile)  
  outputFile=sm"."arr[1]"."lane".fastq"
  outputFile2=sm"."arr[1]"."lane".rgfile"
  print "@RG" tab "ID:"arr[1] tab "PL:illumina" tab "PU:"sm > (outputFile2)
  close(outputFile2)
}
{
  print >> (outputFile)
}' ${FQ_OUT1}

Fixes in OP's attempt:

  • OP needed tab to printed as literal character so I have created -v tab="\\\\t" awk variable to have it in escaping way to be treated it as a literal character.
  • Created new variable for output file named outputFile2 which will create .rgFile for user.
  • That rg file will be having only 1 line because statement is under condition if line starts from @HS.
  • Though rg file will have only 1 line but I am using close to close it in backend for safer side.

Logic of creating output files in awk: This may help anyone to know how to create output files in awk.

  • Always create -v awk_var="$shell_var for keeping it clean to pass shell variables to awk variables. Though there is another of using '" but by this way it looks cleaner to handle.
  • Always close the output file, see whenever awk writes anything to output files it keeps opening them in backend, so its better to close them to avoid error "too many files opened", though new versions of gawk handles that too, but as a good practice you could do it.
user_58665800
Related