How to get file name which have error using AWK command?

user_15970607 0
Viewed 101

I am using the SAC tool to read the header information but some files have no header information and it prints an error. Is there any way to use AWK to print that files if they do not have a header or error during work. I often used AWK for data manipulation but failed this time.

Here is my try:

saclst a f *2020-05*BHZ*

This is the output

GS.GS043.2020-05-18T03:52.BHZ.sac         3.37
GS.GS043.2020-05-18T09:28.BHZ.sac         3.64
GS.GS043.2020-05-18T12:09.BHZ.sac         3.42
saclst: Error determining SAC header: GS.GS043.2020-05-18T14:36.BHZ.sac
GS.GS043.2020-05-18T16:25.BHZ.sac         2.92
GS.GS043.2020-05-18T18:51.BHZ.sac         3.66

Now I want to get the file name and print it but seems like AWK does not help;

saclst a f *2020-05*BHZ* | awk '{if ($2<0) print $1;}' > ../test.dat

My output file is empty and the terminal shows this error:

Is there any way to save this error so I can later modify it?

saclst: Error determining SAC header: SC.LZB.2020-05-21T10:46.BHZ.sac
saclst: Error determining SAC header: SC.LZB.2020-05-21T11:57.BHZ.sac
saclst: Error determining SAC header: SC.LZB.2020-05-26T11:23.BHZ.sac
saclst: Error determining SAC header: SC.LZB.2020-05-28T10:44.BHZ.sac
saclst: Error determining SAC header: SC.QSC.2020-05-12T06:49.BHZ.sac
awk bash
3 Answers

Here's what I think you are looking for:

# just for demo, pipe SAC tool to awk for your actual use case
$ cat ip.txt
GS.GS043.2020-05-18T03:52.BHZ.sac         3.37
GS.GS043.2020-05-18T09:28.BHZ.sac         3.64
GS.GS043.2020-05-18T12:09.BHZ.sac         3.42
saclst: Error determining SAC header: GS.GS043.2020-05-18T14:36.BHZ.sac
GS.GS043.2020-05-18T16:25.BHZ.sac         2.92
GS.GS043.2020-05-18T18:51.BHZ.sac         3.66

# filter lines with Error based on number of fields or `Error` in 2nd field
$ awk 'NF != 2' ip.txt
saclst: Error determining SAC header: GS.GS043.2020-05-18T14:36.BHZ.sac
$ awk '$2 == "Error"' ip.txt
saclst: Error determining SAC header: GS.GS043.2020-05-18T14:36.BHZ.sac

# print only last field
$ awk '$2 == "Error"{print $NF}' ip.txt
GS.GS043.2020-05-18T14:36.BHZ.sac

If the saclst command puts the lines with Error on stderr, you can use this:

$ saclst a f *2020-05*BHZ* 2> error.log
user_40820520

Athough awk (what you asked) works,

sed -n 's/.*Error.*:/ /p' ip.txt

would work fine as well. And

grep Error ip.txt

So, don't focus on using only awk.

user_69088950

if you wanna cleanup that awk code (assuming +$2 is correct), try

saclst ... 2>&1 | [g/m/n]awk '(+$2<+_)*(NF=!_)'

The reason why I'm suggesting the +$2 is that inputs being read in would do a string-based-comparison if that values don't "appear" numeric, leading to the following scenario where invalid data like an exclamation mark ! preceding the numbers in column 2 would end up having the condition evaluate to true, even though it's clearly not a negative number

  • ! is \x21
  • ASCII 0 is \x30

that's why it ended up passing the filter unintentionally :

 echo '12 !232523435 2997 ' \
 \
 | mawk '$2<0'

 12 !232523435 2997

Now that's only assuming OP's original logic of checking $2 being negative is accurate.

To properly handle ip.txt, I tested this other variant instead:

% < ip.txt [g/m/n]awk '/: .+: .+:/ && ($!_=$_=$NF)~_'

GS.GS043.2020-05-18T14:36.BHZ.sac

I created a synthetic version of ip.txt that's 8.27 GB in size. Maybe grep or sed is faster, but personally me thinks a filtering throughput rate of 532 MB/s for awk should be plenty :

 pvE0 <  ip2.txt | mawk2 '/: .+: .+:/ && ($!_=$_=$NF)~_' | pvE9 | xxh128sum 

      in0: 8.27GiB 0:00:15 [ 532MiB/s] [ 532MiB/s]             
     out9:  938MiB 0:00:15 [59.0MiB/s] [59.0MiB/s] [        <=>  ]

e8626c6b7e7c90a5cb21bd00fee6ec41  stdin
user_146721140
Related