Bash command to get start time and endtime from log file

user_1506449 0
Viewed 521

I need to get the start time and end time of from a log file which looks like the below format:

REQUEST     test 1 1651474077633    
.
.
.
.
REQUEST     test n  1651474676561

I want to print start and end time. The time is in Unix format and I want to convert it into standard date and time format.

code which I tried:

cat ${FILENAME} | awk 'BEGIN {
   print strftime("Time = %m/%d/%Y %H:%M:%S", NF)
}'

But it prints all time.

I just want time for 1st entry and last entry.

grep awk unix bash
5 Answers

I think your code uses BEGIN and NF incorrectly.

Try the code below.

awk '$1 == "REQUEST" { print strftime("Time = %m/%d/%Y %H:%M:%S", $4) }' "$FILENAME"

If you want to output only the first and last outputs:

awk '
BEGIN { first = 1 }
$1 == "REQUEST" {
  if (first) {
    firsttime = $4
    first = 0
  }
  lasttime = $4
}
END {
  print strftime("First Time = %m/%d/%Y %H:%M:%S", firsttime)
  print strftime("Last  Time = %m/%d/%Y %H:%M:%S", lasttime)
}' "$FILENAME"

The numbers in the text in question do not seem to be in UNIX time, but you can check there.

user_158215260

They're indeed valid POSIX epoch timestamps, in milliseconds.

Just divide the value by 1000, and you'll get a standard timestamp - May 3rd, 2022:

echo "${ccccc}" |
gawk -be '
  function printtime(_,__,___,____) { 
      print \
          substr((____=RS)(RS="\n"),
          (__=sprintf("gdate +\47%5s Time = "(___\
                 )"\47",__,_*1e-3)) | getline _,
               RS*close(__)^(RS=____))_ 
 } 1<NF {
          __[++_]=$!(NF=NF) 
 }  END { 
          printtime(__[_^!_], "First", ___)
          printtime(__[(+_)],  "Last", ___)

 }' ___='%D %T.%6N'  FS='^REQUEST[^t]+test[ ]+([n]|[0-9]+)[ ]+' | ecp
 
First Time = 05/03/22 00:55:57.458000
 Last Time = 05/03/22 00:55:57.462939
user_146721140

Please find below my approach to the initial question:

Possible solution

$ (head -1 requests.log; tail -1 requests.log) |
  tr -s ' ' |
  cut -d ' ' -f4 |
  awk '{printf( "%.3f\n", $1/1000);}' |
  xargs -I T date --date=@T

Mon, May  2, 2022  9:47:57 AM
Mon, May  2, 2022  9:57:56 AM

Explanation

$ cat requests.log

REQUEST     test 1 1651474077633
REQUEST     test 0 1651474077630
REQUEST     test 1 1651474077631
REQUEST     test 2 1651474077632
REQUEST     test 3 1651474077633
REQUEST     test 4 1651474676561

Step 1 - Get the first and the last lines

$ (head -1 requests.log; tail -1 requests.log)

REQUEST     test 1 1651474077633
REQUEST     test 4 1651474676561

Step 2 - Trim multiple white spaces

$ (head -1 requests.log; tail -1 requests.log) |
tr -s ' '

REQUEST test 1 1651474077633
REQUEST test 4 1651474676561

Step 3 - Get the timestamps

$ (head -1 requests.log; tail -1 requests.log) |
tr -s ' ' |
cut -d ' ' -f4

1651474077633
1651474676561

Step 4 - Convert timestamps to seconds

$ (head -1 requests.log; tail -1 requests.log) |
tr -s ' ' |
cut -d ' ' -f4 |
awk '{printf( "%.3f\n", $1/1000);}'

1651474077.633
1651474676.561

Step 5 - Convert timestamps to human readable dates

$ (head -1 requests.log; tail -1 requests.log) |
tr -s ' ' |
cut -d ' ' -f4 |
awk '{printf( "%.3f\n", $1/1000);}' |
xargs -I T date --date=@T


Mon, May  2, 2022  9:47:57 AM
Mon, May  2, 2022  9:57:56 AM
user_21948430

Checking you timestamps as for example https://www.epochconverter.com/ they seem to be in milliseconds indeed.

If the structure of your file is always as the example file you can take the value of the last field using $NF for the first line and the last line, and print the result in the END block.

If there is a single line that matches, there will be no end time. If there are no lines that match, you can print a custom message.

Note that you don't have to use cat

awk '
!start && $1 == "REQUEST"{
  start = $NF
  next
}

$1 == "REQUEST" {
  last = $NF
}

END {
  if (start) {
    print strftime("Start time = %m/%d/%Y %H:%M:%S", start/1000)
  }
   if (last) {
      print strftime("End time = %m/%d/%Y %H:%M:%S", last/1000)
   } 
   if (!start){
    print "No times available."
  }
}
' "$FILENAME"

Output

Start time = 05/02/2022 08:47:57
End time = 05/02/2022 08:57:56
user_54249880

Suggesting combination of date -d @.... with awk to extract the number.

First date:

date -d @$(awk "{print int(\$4/1000)}" <<< $(head -1 input.txt))
Mon, May  2, 2022  9:47:57 AM

Last date:

date -d @$(awk "{print int(\$4/1000)}" <<< $(tail -1 input.txt))
Mon, May  2, 2022  9:57:56 AM

Time difference in seconds:

echo "secs diff = "$(( $(awk "{print int(\$4/1000)}" <<< $(tail -1 input.txt)) - $(awk "{print int(\$4/1000)}" <<< $(head -1 input.txt)) ))
secs diff = 599
user_62661920
Related