filter dates within a text file

Viewed 97

I have a text file which has a data in the below format. How can we print only those rows which have a date less than 10 days from the current date?

sample.txt

system system_data8 Thu Jul 29 22:36:38 2021
system system_data9 Wed Jan 24 14:43:52 2018
system system_data3 Tue Jan 23 20:25:17 2018
system system_data2 Fri Mar 09 20:37:05 2018
system system_data5 Fri Mar 09 22:02:31 2018

expected output

system system_data8 Thu Jul 29 22:36:38 2021

I was trying something like below but it's not working.

awk -F ' ' '{printf("%s,%s,",$1,$2);"date +%F -d "$3" "$4" "$5" "$6" "$7;}' sample.txt
3 Answers

GNU awk has time functions:

$ gawk -v days=10 'BEGIN {max = days*86400; now = systime()}
NF>3 {
  mn = (index("JanFebMarAprMayJunJulAugSepOctNovDec",$(NF-3)) + 2)/3
  dt = $NF " " mn " " $(NF-2) " " gensub(/:/," ","g",$(NF-1))
  diff = now - mktime(dt)
  if (-max < diff && diff < max)
    print
}' file
system system_data8 Thu Jul 29 22:36:38 2021

Using GNU awk for time functions:

$ cat tst.awk
BEGIN {
    tgtDays = 10
    tgtSecs = tgtDays * 24 * 60 * 60

    endTime = strftime("%Y %m %d 12 00 00")
    endSecs = mktime(endTime,1)
}
{
    mthNr   = (index("JanFebMarAprMayJunJulAugSepOctNovDec",$4)+2)/3
    begTime = sprintf("%04d %02d %02d 12 00 00", $7, mthNr, $5)
    begSecs = mktime(begTime,1)
}
(endSecs - begSecs) < tgtSecs

$ awk -f tst.awk sample.txt
system system_data8 Thu Jul 29 22:36:38 2021

Note that in the above we replace the time of day in both the input data and the current time with noon because when determining how many days between 2 dates by converting a timestamp to seconds since the epoch first then dividing by the number of seconds in a day you have to use the same time each day because otherwise your "number of days" calculation can/will be thrown off by the time each day.

For example look at the following that's trying to determine if 2 dates which ARE 10 days apart are less than 10 days apart:

$ cat diffDatesDemo.awk
BEGIN {
    tgtDays = 10
    tgtSecs = tgtDays * 24 * 60 * 60

    begTime = "2021/08/01 09:00:00"
    endTime = "2021/08/11 08:00:00"

    begDate = gensub(/([ :][0-9]{2}){3}$/,"",1,begTime)
    endDate = gensub(/([ :][0-9]{2}){3}$/,"",1,endTime)

    print "Is", begTime, "less than", tgtDays, "days before", endTime "?"

    ####
    print "\nWrong: Compare 2 timestamps including date plus time of day:"
    begSecs = mktime(gensub("[/:]"," ","g",begTime),1)
    endSecs = mktime(gensub("[/:]"," ","g",endTime),1)

    print begDate, "->", endDate, "is", ((endSecs - begSecs) < tgtSecs ? "<" : ">="), tgtDays, "days"
    ####

    ####
    print "\nRight: Compare 2 dates at the same time each day:"
    begSecs = mktime(gensub("[/:]"," ","g",begDate)" 12 00 00",1)
    endSecs = mktime(gensub("[/:]"," ","g",endDate)" 12 00 00",1)

    print begDate, "->", endDate, "is", ((endSecs - begSecs) < tgtSecs ? "<" : ">="), tgtDays, "days"
    ####
}

$ awk -f diffDatesDemo.awk
Is 2021/08/01 09:00:00 less than 10 days before 2021/08/11 08:00:00?

Wrong: Compare 2 timestamps including date plus time of day:
2021/08/01 -> 2021/08/11 is < 10 days

Right: Compare 2 dates at the same time each day:
2021/08/01 -> 2021/08/11 is >= 10 days

I also used the UTC flag for mktime() above to make sure that any local DST changes didn't impact the number of days calculation.

If your date utility is recent enough and if your shell is bash you can do this only with bash and date:

now=$(date +%s)
while read sys sys_d dt; do
  sec=$(date -d "$dt" +%s)
  if (( now-sec <= 10*24*3600 )); then
    echo "$sys $sys_d $dt"
  fi
done < sample.txt

Note that the comparison is done after converting the current date/time to a UNIX timestamp (date +%s), that is, seconds since 1970-01-01. Same for the date/time in your file. This does not account for date/time irregularities like daylight saving time or leap seconds. So, depending on when you run this, on the content of your file and on your definition of 10 days, the result may be what you want or not exactly what you want.

Related