Get the last 5 minutes from log file

Viewed 435

Im trying to get the last 5 minutes from a log file where time format is like this:

2021-08-10 16:05:00,007 ERROR [com.] Cought an Exception (el) Index: 1, Size: 0
2021-08-10 16:05:00,018 ERROR [com.] Cought an Exception (el) Index: 2, Size: 0
2021-08-10 16:10:00,005 ERROR [com.] Cought an Exception (el) Index: 2, Size: 0
2021-08-10 16:15:00,002 ERROR [com.] Cought an Exception (el) Index: 1, Size: 0
2021-08-10 16:15:00,014 ERROR [com.] Cought an Exception (el) Index: 2, Size: 0
2021-08-10 16:50:00,008 ERROR [com.] Cought an Exception (el) Index: 2, Size: 0

I have tried below sed command which is working on another format but I get nothing returned when i run this.

sed -n "/^$(date --date='5 minutes ago' '+%Y-%m-%d %H:%M')/,\$p" logfile

Tried many many different commands but cannot find one to work for this time format 2021-08-10 16:50:00,008

Anyone help is really appreciated.

Also now I'm trying with awk but its getting some random lines which are not on the last minutes but somewhere on the logfile: (i only want to get the last X minutes)

awk -v dt="$(date '+%Y-%m-%d %T,%3N' -d '-30 minutes')" '$1 " " $2 > dt' logfile
Raw Trap [OID=.2.6.1.4.1.5.1.2.4...
Raw Trap [OID=.2.6.1.4.1.5.1.2.4...
Raw Trap [OID=.2.6.1.4.1.5.1.2.14...
Raw Trap [OID=.2.6.1.4.1.5.1.2.14,...
Raw Trap [OID=.2.6.1.4.1.5.1.2.14...
Raw Trap [OID=.2.6.1.4.1.5.1.2.14...
Raw Trap [OID=.2.6.1.4.1.5.1.2.14,...
Raw Trap [OID=.2.6.1.4.1.5.1.2.14,....
2021-08-11 14:00:00,010 ERROR [com.] Cought an IOException (el) Index: 2, Size: 0
2021-08-11 14:05:00,010 ERROR [com.] Cought an IOException (el) Index: 2, Size: 0`

more input from logfile:

2021-08-06 14:24:16,137 INFO  [com.] Invoking Rule: (class path...)
Raw Trap [OID=.1.3.6.1.2.14, Sender=v, VarBind OID=.1.3.6.3, VarBind Value=0] --> Clean Trap [........
2021-08-06 14:24:16,137 INFO  [com.] InstanceStr=null
2021-08-06 14:24:16,137 INFO  [com.] site_InstanceStr=null
2021-08-06 14:24:16,141 INFO  [com.] deleting raw trap com.RawTrapid: 7291, oid: .1.3.6.1.4, time: 2021/08/06 14:2...
2021-08-07 00:30:12,495 INFO  [com.] Exporting with user Admin
2021-08-07 00:30:12,511 INFO  [com.] preparing for bundle export; retrieving all parameters with bundle id matching: 
2021-08-07 00:31:07,538 INFO  [com.] Exporting with user Admin
2021-08-07 00:31:07,538 INFO  [com.] preparing for bundle export; retrieving all parameters with bundle id matching: 
2021-08-07 00:31:07,573 INFO  [com.] parameters retrieved: 1001
2021-08-07 00:31:07,573 INFO  [com.] creating temp bundle export directory in: /var/tmp/_export2021/fullversion
2021-08-07 00:31:07,914 INFO  [com.] Deleting temp dir: /var/tmp/_export_Sat_Aug_07_00_31_07_CEST_2021
2021-08-07 04:00:00,115 ERROR [com.] Partition maintenance failed: exit code=1, output: ERROR 1507 (HY000) at line 26: Error in list of partitions to DROP
1 Answers

You can compute date in required format in the shell and pass it as argument to awk to conpare:

awk -v dt="$(date '+%Y-%m-%d %T' -d '-5 minutes')" -F, '$1 > dt' file.log

If you want to include milli-second part also then use:

awk -v dt="$(date '+%Y-%m-%d %T,%3N' -d '-5 minutes')" '($1 " " $2) > dt' file

PS: This requires gnu-date.

Related