Format data to table format

user_13081396 0
Viewed 76

I am a total novice to coding, but am wondering the easiest way to generate a table from grep count data.

My grep count output file looks like this:

AAR34355.1
./006D_id70.m8:0
./20D_id70.m8:0
./28D_id70.m8:0
AAR38850.1
./006D_id70.m8:0
./20D_id70.m8:2
./28D_id70.m8:4
A13520.1
./006D_id70.m8:0
./20D_id70.m8:0
./28D_id70.m8:0

I need an output to look more like this:

            ./006D_id70.m8    ./20D_id70.m8    ./28D_id70.m8
AAR34355.1         0                0                 0
AAR38850.1         0                2                 4
A13520.1           0                0                 0

or at least a delimited equivalent.

Forgive my description, as I am pretty new to this.

Is there a relatively simple way of formatting the data this way?

bash
2 Answers

You can do that all in awk, no need to reshape grep's output. Assuming patterns to be searched for are listed in a file named patterns, and files to be searched in are file1, file2, and file3; copy and save the following code block into a file named tst.awk,

NR == FNR {
  pat[NR] = $0
  next
}

FNR == 1 {
  fil[c++] = FILENAME
}

{
  for (i in pat)
    if ($0 ~ pat[i])
      mat[FILENAME, pat[i]]++
}

END {
  for (i in fil)
    printf "\t%s", fil[i]

  print ""

  for (i in pat) {
    printf "%s", pat[i]

    for (j in fil)
      printf "\t%d", mat[fil[j], pat[i]]

    print ""
  }
}

and run

awk -f tst.awk patterns file1 file2 file3

Demo:

$ seq 5 > file1
$ seq 3 7 > file2
$ seq 5 9 > file3
$ seq 3 2 7 > patterns
$ awk -f tst.awk patterns file1 file2 file3
        file1   file2   file3
3       1       1       0
5       1       1       1
7       0       1       1
user_102486780

OP's script first but in more reader friendly form

#!/bin/bash
while read   protein; do
    echo    $protein
    grep -c $protein ./*.m8
done < test_just_IDs.txt

Will give

$ ./test
AAR34355.1
./006D_id70.m8:0
./20D_id70.m8:0
./28D_id70.m8:0
AAR38850.1
./006D_id70.m8:0
./20D_id70.m8:2
./28D_id70.m8:4
A13520.1
./006D_id70.m8:0
./20D_id70.m8:0
./28D_id70.m8:0

And if we modify it a bit

#!/bin/bash
printf "\t"; ls ./*.m8
while read   protein; do
    printf "$protein\t"
    printf "%s\t\t" $(grep -hc $protein ./*.m8)
    echo
done < test_just_IDs.txt

It will give desired output

$ ./test
        ./006D_id70.m8  ./20D_id70.m8  ./28D_id70.m8
AAR34355.1      0               0               0       
AAR38850.1      0               2               4       
A13520.1        0               0               0

And with something like this(for tab 8)

#!/bin/bash
line='%16s';        printf "\t\t"
for file in *.m8; { printf "| %s\t" $file; line+='+%15s'; }
while read   protein; do
    printf "\n$line\n" | tr " " "-"
    printf "$protein\t"
    printf "|\t%s\t" $(grep -ch $protein *.m8)
done < test_just_IDs.txt

We'll get a real table look

$ ./test
                | 006D_id70.m8  | 20D_id70.m8   | 28D_id70.m8   
----------------+---------------+---------------+---------------
AAR34355.1      |       0       |       0       |       0   
----------------+---------------+---------------+---------------
AAR38850.1      |       0       |       2       |       4   
----------------+---------------+---------------+---------------
A13520.1        |       0       |       0       |       0
user_126074430
Related