Why fgrep is acting weird?

user_8376143 0
Viewed 106

I am trying to use grep to extract the matching words from file1 in file2 and directing to output file.

I have file1 

Acetoanaerobium sticklandii
Acetobacter pasteurianus

like >1000 species.

File2 

>WP_013360383.1 ATP-dependent Clp protease ATP-binding subunit [Acetoanaerobium sticklandii]
>WP_013360396.1 ATP-dependent Clp protease ATP-binding subunit [Acetoanaerobium sticklandii]
>WP_003623694.1 multidrug efflux RND transporter permease subunit [Acetobacter pasteurianus]
>WP_003624003.1 superoxide dismutase [Acetobacter pasteurianus]
>WP_003624029.1 UDP-galactopyranose mutase [Acetobacter pasteurianus]
>WP_003624540.1 mannose-1-phosphate guanylyltransferase/mannose-6-phosphate isomerase [Acetobacter pasteurianus]
>WP_077905956.1 effector protein [Salmonella enterica]
>WP_077905962.1 type III secretion system YopJ family effector AvrA [Salmonella enterica]
>WP_005544680.1 3-deoxy-8-phosphooctulonate synthase [Aggregatibacter actinomycetemcomitans]
>WP_005545812.1 MFS transporter [Aggregatibacter actinomycetemcomitans]
>WP_005546163.1 UTP--glucose-1-phosphate uridylyltransferase GalU [Aggregatibacter actinomycetemcomitans]

and so on..

when I am using the code 

grep -f file1 file2 > output

or 

fgrep -f file1 file2 > output

The resulting output is the list containing the last line of file1 and remaining input list is ignoring by grep. Even with -w option also the output is same.

Why the grep is behaving like this? Is it problem with my shell? I am using MacBook with Mojave OS version.

Please suggest me equivalent awk commands.

I tried 

awk 'NR==FNR{a[$0];next}$NF in a{print}' file1 file2 > output

but resulted empty file.

grep macos awk
2 Answers

1st solution: Generic and faster solution than my 2nd solution here. This will look for string from [ to ] in Input_file2 irrespective of field's hard-coding here.

awk '
{ gsub(/\r/,"") }
FNR==NR{
  array[$0]
  next
}
match($0,/\[[^]]*/){
  val=substr($0,RSTART+1,RLENGTH-1)
}
(val in array)
' file1 file2

Explanation: Added detailed explanation for above.

awk '                                     ##Starting awk program from here.
{ gsub(/\r/,"") }
FNR==NR{                                  ##Checkiing condition FNR==NR which will be TRUE hen file1 is being read.
  array[$0]                               ##Creating array with index of current line.
  next                                    ##next will skip all statements from here.
}
match($0,/\[[^]]*/){                      ##Using match to match from [ to till ] in line.
  val=substr($0,RSTART+1,RLENGTH-1)       ##Creating val which has sub-string from RSTART  to RLENGH here.
}
(val in array)                            ##Checking condition if val is present in array then print that line.
' file1 file2                             ##Mentioning Input_file names here.

2nd solution: Could you please try following, written and tested with shown samples. This is more generic solution since field number is NOT hard coded in solution but this code will work irrespective of field numbers here.

awk '{ gsub(/\r/,"") } FNR==NR{array[$0];next} {for(i in array){if(match($0,i)){print;next}}}' file1 file2

Explanation: Adding detailed explanation for above.

awk '                    ##Starting awk program from here.
{ gsub(/\r/,"") }
FNR==NR{                 ##Checking condition FNR==NR if that's true then do following.
  array[$0]              ##Creating an array with index of current line.
  next                   ##next will skip further statements from here.
}
{
  for(i in array){       ##Looping through array here.
    if(match($0,i)){     ##Checking if current key is present  in current line then do following.
      print              ##Printing current line here.
      next               ##next will skip further statements from here.
    }
  }
}
'  file1 file2           ##Mentioning Input_file names here.
user_58665800

The problem is with your match condition $NF in a, means to match sticklandii] to your full line content in first file.

A better approach would be to set delimiter to process the whole string inside [..] in second file against first file

awk 'NR == FNR{ a[$0]; next } $2 in a' f1 FS='[][]' f2

The FS='[][]' before second file is to set the field separator to [] which means you can access the entries within [..] by column numbers which is $2 in your given case.

As for your original problem fgrep not working, you need to add the -F flag to make the strings from second file as "fixed" strings and not regex strings. So you can do

fgrep -Ff file1 file2
user_52910150
Related