Extract common lines from multiple text files and display original line numbers

user_8435047 0
Viewed 150

What I want?

  1. Extract the common lines from n large files.
  2. Append the original line numbers of each files.

Example:

File1.txt has the following content

apple
banana
cat

File2.txt has the following content

boy
girl
banana
apple

File3.txt has the following content

foo
apple
bar

The output should be a different file

1 3 2 apple

1, 3 and 2 in the output are the original line numbers of File1.txt, File2.txt and File3.txt where the common line apple exists

I have tried using grep -nf File1.txt File2.txt File3.txt, but it returns

File2.txt:3:apple
File3.txt:2:apple
grep awk shell
4 Answers

Associate each unique line with a space separated list of line numbers indicating where it is seen in each file in an array, and print these next to each other at the end if the line is found in all three files.

awk '{
  n[$0] = n[$0] FNR OFS
  c[$0]++
}
END {
  for (r in c)
    if (c[r] == 3)
      print n[r] r
}' file1 file2 file3

If the number of files is unknown, refer to Ravinder's answer, or just change the hardcoded 3 in the END block with ARGC-1 as shown there.

user_102486780

GNU awk specific approach that works with any number of files:

#!/usr/bin/gawk -f
BEGINFILE {
    nfiles++
}
{
    lines[$0][nfiles] = FNR
}
END {
    PROCINFO["sorted_in"] = "@ind_str_asc"
    for (line in lines) {
        if (length(lines[line]) == nfiles) {
            for (file = 1; file <= nfiles; file++)
                printf "%d\t", lines[line][file]
            print line
        }
    }
}

Example:

$ ./showlines file[123].txt
1   3   2   apple
user_99521960

Could you please try following, written and tested with GNU awk, one could make use of ARGC value which gives us total number of element passed to awk program.

awk '
{
  a[$0]=(a[$0]?a[$0] OFS:"")FNR
  count[$0]++
}
END{
  for(i in count){
    if(count[i]==(ARGC-1)){
       print i,a[i]
    }
  }
}
' file1.txt file2.txt file3.txt
user_58665800

A perl solution

perl -ne '
  $h{$_} .= "$.\t";  # append current line number and tab character to value in a hash with key current line
  $. = 0 if eof;     # reset line number when end of file is reached
  END{
    while ( ($k,$v) = each %h ) { # loop over has entries
      if ( $v =~ y/\t// == 3 ) {  # if value contains 3 tabs
        print $v.$k               # print value concatenated with key
      }
    }
  }' file1.txt file2.txt file3.txt
user_14547080
Related