grep for a particular string and count the number of fatals and errors

user_13640208 0
Viewed 75

I am having a file called violations.txt as below:

column1                  column2      column3         column4                    Situation
Data is preesnt         |  Bgn      | Status (!)    | There are no current runs | Critical level
Data is not existing    |  Nbgn     | Status (*)    | There are runs            | Medium level
Data limit is exceeded  |  Gnp      | Status (!)    | The runs are not present  | Higher level
Dats existing|present   |  Esp      | Status (*)    | The runs are present      | Normal|Higher level

I need the output like this:

violations.txt:
Fatal:
    Bgn : 1
    Gnp : 1
Total number of fatals : 2  
Errors:
    Nbgn : 1
    Esp  : 1
Total number of errors : 2

I am trying to execute if the file violations.txt conatins in the column3 the word Status (!) as a fatal and if it contains the word Status(*) as a warning and also the count of it.

I tried the below code but not getting the exact output:

#!/bin/bash
pwd
echo " " ;
File="violations.txt"
for g in $File;
do
awk -F' +\\| +' 
if "$3"== "Status (!) /"  "$File" ; then
'BEGIN{ getline; getline }
truncate -s -1 "$File"
echo "$g:";
         { a[$2]++ }
     END{ for(i in a){ print i, a[i]; s=s+a[i] };
        print "Total numer of fatals:", s}' violations.txt
else 
echo "$g:";
'BEGIN{ getline; getline }
truncate -s -1 "$File"
echo "$g:";
         { a[$2]++ }
     END{ for(i in a){ print i, a[i]; s=s+a[i] };
        print "Total numer of errors:", s}' violations.txt
fi
done
awk regex
4 Answers

Haven't we already covered this in a somewhat different reincarnation?

$ cat tst.awk

BEGIN {
  FS="[[:blank:]][|][[:blank:]]"
  OFS=" : "
}
FNR>1{
   gsub(/[[:blank:]]/, "", $2)
   gsub(/[[:blank:]]/, "", $3)
   a[$3][$2]++
}
END {
  #PROCINFO["sorted_in"]="@ind_str_desc"
  print "Out" OFS
  for(i in a) {
    print ($i~/*/?"Fatal":"Error") OFS
    t=0
    for(j in a[i]) {
      print "\t" j, a[i][j]
      t+=a[i][j]
    }
    print "Total", t
    t=0
  }
}

running awk -f tst.awk myFile results in:

Out :
Fatal :
        Gnp : 1
        Bgn : 1
Total : 2
Fatal :
        Esp : 1
        Nbgn : 1
Total : 2
user_136805340

Could you please try following, written and tested with shown samples. Written and tested in https://ideone.com/rsVIV4

awk '
BEGIN{
  FS="\\|"
}
FNR==1{  next  }
/Status \(\!\)/{
  match($0,/\| +[a-zA-Z]+ +\| Status/)
  val=substr($0,RSTART,RLENGTH)
  gsub(/\| +| +\| Status/,"",val)
  countEr[val]++
  val=""
}
/Status \(\*\)/{
  match($0,/\| +[a-zA-Z]+ +\| Status/)
  val=substr($0,RSTART,RLENGTH)
  gsub(/\| +| +\| Status/,"",val)
  countSu[val]++
  val=""
}
END{
  print "Fatal:"
  for(i in countEr){
     print "\t"i,countEr[i]
     sumEr+=countEr[i]
  }
  print "Total number of fatal:"  sumEr
  for(i in countSu){
     print "\t"i,countSu[i]
     sumSu+=countSu[i]
  }
  print "Total number of errors:"sumSu
}
' Input_file

Explanation: Adding detailed explanation for above.

awk '                                       ##Starting awk program from here.
BEGIN{                                      ##Starting BEGIN section from here.
  FS="\\|"                                  ##Setting field separator as | for all lines here.
}
FNR==1{  next  }                            ##Checking condition if FNR==1 then go next and do not do anything on this line.
/Status \(\!\)/{                            ##Checking condition if line contains Status (!) then do following.
  match($0,/\| +[a-zA-Z]+ +\| Status/)      ##Using match function to match pipe space letters space and | space and Status string here.
  val=substr($0,RSTART,RLENGTH)             ##Creating sub-string from current line here.
  gsub(/\| +| +\| Status/,"",val)           ##Globally substituting pipe space and Status keyword with NULL in val here.
  countEr[val]++                            ##Creating array countEr with index of val and increment its count with 1 here.
  val=""                                    ##Nullifying val here.
}
/Status \(\*\)/{                            ##Checking condition if line contains Status (*) then do following.
  match($0,/\| +[a-zA-Z]+ +\| Status/)      ##Using match function to match pipe space letters space and | space and Status string here.
  val=substr($0,RSTART,RLENGTH)             ##Creating sub-string from current line here.
  gsub(/\| +| +\| Status/,"",val)           ##Globally substituting pipe space and Status keyword with NULL in val here.
  countSu[val]++                            ##Creating array countSu with index of val and increment its count with 1 here.
  val=""                                    ##Nullifying val here.
}
END{                                        ##Starting END block of this program from here.
  print "Fatal:"                            ##Printing Fatal keyword here.
  for(i in countEr){                        ##Traversing through countEr here.
     print "\t"i,countEr[i]                 ##Printing tab i and value of countEr with index i here.
     sumEr+=countEr[i]                      ##Creating sumEr and keep adding value of countEr here.
  }
  print "Total number of fatal:"  sumEr     ##Printing string Total number of fatal/l and value of sumEr here.
  for(i in countSu){                        ##Traversing through countSu here.
     print "\t"i,countSu[i]                 ##Printing tab i and value of countSu with index i here.
     sumSu+=countSu[i]                      ##Creating sumSu and keep adding value of countSu here.
  }
  print "Total number of errors:"sumSu      ##Printing string Total number of errors: with value of sumSu here.
}
' Input_file                                ##Mentioning Input_file name here.
user_58665800

With GNU awk for various extensions and using the fact that your input is fixed-width fields:

$ cat tst.awk
BEGIN {
    FIELDWIDTHS="24 1 11 1 15 1 27 1 *"
}
NR>1 {
    type = ($5 ~ /!/ ? "Fatal" : "Error")
    keyTot[type][gensub(/\s/,"","g",$3)]++
    tot[type]++
}
END {
    for (type in tot) {
        print type ":"
        for (key in keyTot[type]) {
            print "    " key " : " keyTot[type][key]
        }
        print "Total number of " type " : " tot[type]+0
    }
}

.

$ awk -f tst.awk file
Error:
    Esp : 1
    Nbgn : 1
Total number of Error : 2
Fatal:
    Gnp : 1
    Bgn : 1
Total number of Fatal : 2
user_17450010

Your file looks very badly formatted, from a computer point of view, let me explain you why:

column1                  column2      column3         column4                    Situation
Data is preesnt         |  Bgn      | Status (!)    | There are no current runs | Critical level
Data is not existing    |  Nbgn     | Status (*)    | There are runs            | Medium level
Data limit is exceeded  |  Gnp      | Status (!)    | The runs are not present  | Higher level
Dats existing|present   |  Esp      | Status (*)    | The runs are present      | Normal|Higher level

The locations of the first character of the headers of columns 1, 3 and 4 are equal to the first characters of the contents, but for columns 2 and 5, this is not the case.
You are using the pipe character "|" as a separator for your columns, but also for a separator within the columns themselves. This combination is very bad for automatic parsin, based on "|" character as a separator.

Therefore I have following proposals for improving your file:

First let's take care of the column headings first characters:

column1                    column2    column3         column4                     Situation
Data is preesnt         |  Bgn      | Status (!)    | There are no current runs | Critical level
Data is not existing    |  Nbgn     | Status (*)    | There are runs            | Medium level
Data limit is exceeded  |  Gnp      | Status (!)    | The runs are not present  | Higher level
Dats existing|present   |  Esp      | Status (*)    | The runs are present      | Normal|Higher level

If you agree on this, you might use the amount of characters for reading your columns.

Second, let's change the internal separator (replace it by a slash character):

column1                   column2     column3         column4                     Situation
Data is preesnt         |  Bgn      | Status (!)    | There are no current runs | Critical level
Data is not existing    |  Nbgn     | Status (*)    | There are runs            | Medium level
Data limit is exceeded  |  Gnp      | Status (!)    | The runs are not present  | Higher level
Dats existing/present   |  Esp      | Status (*)    | The runs are present      | Normal/Higher level

Do you agree with my first or second proposal? If yes, please adapt your question (by adding the agreed proposal), this will make everything easier to handle.

user_42791550
Related