How to get lines from log file from last 10 minutes with specific string

Viewed 136

Tried other solution but not giving correct solutions my time format is [Thu Aug 20 09:28:51 2020]. Most close one was this one

  awk -vDate=`date -d'now-2 hours' +[%a %b %d %H:%M:%S %Y]` '$4 > Date {print Date, $0}' $input

my log file are like this

 [Thu Aug 20 09:10:51 2020] [error] vendor
 [Thu Aug 20 09:23:51 2020] [error] vendor
 [Thu Aug 20 09:25:51 2020] [error] vendor
 [Thu Aug 20 09:27:51 2020] [error] vendor
 [Thu Aug 20 09:28:51 2020] [error] dad

i want result as from current time [Thu Aug 20 09:28:51 2020] to last 10 mins

  [Thu Aug 20 09:23:51 2020] [error] vendor
 [Thu Aug 20 09:25:51 2020] [error] vendor
 [Thu Aug 20 09:27:51 2020] [error] vendor
 [Thu Aug 20 09:28:51 2020] [error] dad
2 Answers

well i tried doing directly with grep but i dont know why but the grep wasnt taking this date format and give some wrong output so i did some way around for it .

#!/bin/bash
input="/home/babin/Desktop/code2"
count=0

dateyear=$(date +'%Y')
month=$(date +'%b')
day=$(date +'%a')

#do loop for 10 mins from now
for (( i = 0; i <=9; i++ )) ; do
     if grep $(date +%R -d "-$i  min") $input | grep -i "error" | grep -wi "$month" | grep -wi "$year" | grep -wi "$day"
     then
        currentcount=$(grep $(date +%R -d "-$i  min") $input | grep -wi "70007" | grep -wi "$month" | grep -wi "$year" | grep -wic "$day")
     else
        currentcount=0
        echo "not found"
     fi
      count=$(( $count + $currentcount )) 
done
    echo "$count"
  #check no of error found and  do task
 if(( $count >= 10))
    then
    echo "more oe equal to 10 finds"
   else
    echo  "less than 10 occurence"
   fi

it gives output as currenttime is [Thu Aug 20 09:28:51 2020] also it matches "error" string.

  enter   [Thu Aug 20 09:23:51 2020] [error] vendor
 [Thu Aug 20 09:25:51 2020] [error] vendor
 [Thu Aug 20 09:27:51 2020] [error] vendor
 [Thu Aug 20 09:28:51 2020] [error] dadcode here

The overall flow is:

  1. Preprocess input to exctract the date part
  2. Convert date to seconds since epoch
  3. Filter seconds since epoch according to the conditions given
  4. Remove seconds since epoch.
  5. Output.

As a overall rule, work in bash using streams. The strptime is from dateutils package. Like so:

# Extract the date+time part from within [..] and put it on the first column with tab
sed 's/ \[\([^]]*\)\]/\1\t&/' "$input" |
# For each line
while IFS=$'\t' read -r date rest; do
    # Convert the date to seconds since epoch
    date=$(strptime -f "%s" -i "%a %b %d %H:%M:%S %Y" "$date")
    # Output the updated line
    printf "%s\t%s\n" "$date" "$rest"
done |
# Read it all in awk and compare second since epoch in the first field to given value
awk -v "since=$(date -d'now -2 hours' +%s)" '$1 > since' |
# Remove first field - ie. second since epoch
cut -f2-

Do not use backticks ``. They are discouraged. Use $(...) instead. Remember as a rule of a thumb to quote all variable expansions. Check your scripts for most common mistakes with http://shellcheck.net . I think somewhere between date and strptime you may encounter problems with your timezone (ie difference in the number of hours).

Related