How to combine .tab.gz files into one gz file not repeating the colunm header?

user_12868740 0
Viewed 180

I am currently using this simple cat to combine gzip tab-delimited files. However, since both have the same column header, I am ending up with one header at the beginning of the file and another header at the middle, where it was concatenated. How can I leave just one header for the c.gz file without having to decompress the files?

Example of the files:

a.tab

col1      col2      col3
1      2      3

b.tab

col1      col2      col3
1      4      6

desired c.tab

col1      col2      col3
1      2      3
1      4      6

What I am trying: cat a.tab.gz b.tab.gz > c.tab.gz

What I am getting in the c.tab:

col1      col2      col3
1      2      3
col1      col2      col3
1      4      6
shell command-line bash
2 Answers

Using awk:

$ awk 'FNR>1||NR==1' <(gunzip -c a.tab.gz) <(gunzip -c b.tab.gz) | gzip > c.tab.gz

Output inside c.tab.gz:

col1      col2      col3
1      2      3
1      4      6

Edit: Another awk:

$ zcat [ab].tab.gz | awk 'NR==1{h=$0;print}$0!=h' | gzip > c.tab.gz

which excludes the records which are identical to the first record of the first file uncompressed - which might cause problems if you have headers in the data.

user_41623560 
(zcat a.tab.gz;awk 'NR>1 { print }' <(zcat b.tab.gz)) | gzip > c.tab.gz

Cat the contents of a.tab.gz with zcat and then redirect the output of zcat b.tab.gz back into awk, printing everything apart from the first line. Pipe the combined output to gzip wand the file c.tab.gz

Same solution with sed processing headers:

(zcat a.tab.gz;sed -n '2,$p' <(zcat b.tab.gz)) | gzip > c.tab.gz
user_78404400
Related