I have to process a input file, if it has more than 1 record (basically any row other than the header).
Parse the 1st and 3rd columns to see if it has the string "Test" and exclude those rows.
Then, write the output - preferably to the same file again.
I searched on SE and Google and see this as very close option : Using `awk` to print number of lines in file in the BEGIN section and the comments from Ed Morton on the same thread.
I have it working in parts, but need to bring everything together.
for test setup I created 15 rows in the input file.
$awk 'NR==FNR {next} FNR==1 {x=NR-1} { print x}' {filename.csv,filename.csv}
15
15
15
15
15
15
15
15
15
15
15
15
15
15
15
Which is validation that x is able to hold the row count.
I was hoping, the below would process all the 15 rows in my input file now that x is holding 15 - however it just seems to processes one row.
$awk 'NR==FNR {next} FNR==1 {x=NR-1} { if(x>1){print x;exit 0;}}' {filename.csv,filename.csv}
15
Here is the input file structure and the expected output.
before the process
filename.csv
col1,col2,col3,.....coln
test,xxx,test,.....
test,xxx,xxx,.....
xxx,xxx,test,.....
yyy,yyy,yyy,.....
zzz,zzz,zzz,.....
after the process
filename.csv
col1,col2,col3,.....coln
yyy,yyy,yyy,.....
zzz,zzz,zzz,.....
This part to perform my check on column1 and column3
if ($1 !~ /[Tt][Ee][Ss][Tt]/ && $3 !~ /[Tt][Ee][Ss][Tt]/) {print $0} else exit 0}
P.S.
-1 File will have atleast the header record.
-2 getline is not available
-3 could not install sponge from moreutils to redirect to same file.
-4 Planning to use tee for redirection to same file.
all of the code | tee filename.csv