Bash/AWK conditionals using two files

Question

First of all, thank you for your help. I have a problem trying to use bash conditionals using two files. I have the file letters.txt

A
B
C

And I have the file number.txt

And I want to use conditionals so if one letter in file letter.txt is also in number.txt it generates the file a.txt b.txt c.txt so the will look as this:

a.txt

A  12
A  15

b.txt

 B  10
 B  14

c.txt

C  18

I know I can do it using this code:

cat number.txt | awk '{if($1=="A")print $0}' > a.txt

But I want to do it using two files.

user_1745001 · Answer

The efficient way to approach this type of problem is to sort the input on the key field(s) first so you don't need to have multiple output files open simultaneously (which has limits and/or can slow processing down managing them) or be opening/closing output files with every line read (which is always slow).

Using GNU sort for -s (stable sort) to retain input order of the non-key fields and only having 1 output file open at a time and keeping it open for the whole time it's being populated:

$ sort -k1,1 -s number.txt |
awk '
    NR==FNR { lets[$1]; next }
    !($1 in lets) { next }
    $1 != prev { close(out); out=tolower($1) ".txt"; prev=$1 }
    { print > out }
' letters.txt -

$ head ?.txt
==> a.txt <==
A  15
A  12

==> b.txt <==
B  10
B  14

==> c.txt <==
C  18

If you don't have GNU sort for -s to retain input order of the lines for each key field, you can replace it with awk | sort | cut, e.g.:

$ sort -k1,1 -s number.txt
A  15
A  12
B  10
B  14
C  18
D  20
E  23

$ awk '{print NR, $0}' number.txt | sort -k2,2 -k1,1n | cut -d' ' -f2-
A  15
A  12
B  10
B  14
C  18
D  20
E  23

Note the change in the order of the 2nd fields for A compared to the input order without doing the above because by default sort doesn't guarantee to retain the relative line order for each key it sorts on:

$ sort -k1,1 number.txt
A  12
A  15
B  10
B  14
C  18
D  20
E  23

user_5866580 · Answer

With your shown samples, please try following.

awk '
FNR==NR{
  arr[$0]
  next
}
($1 in arr){
  outputFile=(tolower($1)".txt")
  print >> (outputFile)
  close(outputFile)
}
' letters.txt numbers.txt

Explanation: Adding detailed explanation for above.

awk '                      ##Starting awk program from here.
FNR==NR{                   ##Checking condition which will be TRUE when letters.txt is being read.
  arr[$0]                  ##Creating array arr with index of current line.
  next                     ##next will skip all further statements from here.
}
($1 in arr){               ##checking condition if 1st field is present in arr.
  outputFile=(tolower($1)".txt")    ##Creating outputFile to print output.
  print >> (outputFile)    ##Printing current line into output file.
  close(outputFile)        ##Closing output file in backend.
}
' letters.txt numbers.txt  ##Mentioning Input_file names here.

Bash/AWK conditionals using two files

2 Answers