Get a record based on Updated time and Severity

Viewed 123

I have some event data, I need to print only single data with respect to a single event based on the Updated time of the event.

I have all the data saved in the hash of arrays and from there I need to extract a single record from each of the events which have the latest updated time.

Here is the script:

use strict;
use warnings;

my %data = (
          '1342' => [
                      {
                        'Severity' => 'MEDIUM',
                        'Node' => 'Node002',
                        'State' => 'ACTIVE_UNACKNOWLEDGED',
                        'Updated' => '2020-10-01T12:00:00',
                        'eventId' => '1342'
                      },
                      {
                        'Severity' => 'HIGH',
                        'Node' => 'Node002',
                        'Updated' => '2020-10-01T12:05:00',
                        'eventId' => '1342',
                        'State' => 'ACTIVE_UNACKNOWLEDGED'
                      }
                    ],
          '1341' => [
                      {
                        'State' => 'ACTIVE_UNACKNOWLEDGED',
                        'eventId' => '1341',
                        'Updated' => '2020-10-01T12:10:00',
                        'Node' => 'Node001',
                        'Severity' => 'HIGH'
                      },
                      {
                        'State' => 'ACTIVE_UNACKNOWLEDGED',
                        'Updated' => '2020-10-01T12:15:00',
                        'eventId' => '1341',
                        'Severity' => 'HIGH',
                        'Node' => 'Node001'
                      },
                      {
                        'State' => 'CLEARED_ACKNOWLEDGED',
                        'Updated' => '2020-10-01T12:15:00',
                        'eventId' => '1341',
                        'Severity' => 'CLEARED',
                        'Node' => 'Node001'
                      }
                    ]
);

foreach my $id ( sort keys %data ){
    print "Event: $id\n";
    foreach my $record( sort { $b->{Updated} cmp $a->{Updated} } @{ $data{$id} }) {
        print "\t";
        print "$record->{Node},$record->{Severity},$record->{State},$record->{Updated}";
        print "\n";
        last;
    }
}

Current Output:

Event: 1341
        Node001,HIGH,ACTIVE_UNACKNOWLEDGED,2020-10-01T12:15:00
Event: 1342
        Node002,HIGH,ACTIVE_UNACKNOWLEDGED,2020-10-01T12:05:00

Expected Output:

Event: 1341
        Node001,CLEARED,CLEARED_ACKNOWLEDGED,2020-10-01T12:15:00
Event: 1342
        Node002,HIGH,ACTIVE_UNACKNOWLEDGED,2020-10-01T12:05:00

If the particular event has the same updated time for different severities, it should pick the record which has CLEARED severity.

In the above case, we can see for EventID 1341 there are CLEARED and HIGH severity exists at the same time i.e., 2020-10-01T12:15:00. So, I should pick the CLEARED severity record instead of HIGH severity. How can I pick that record?

2 Answers

Iterate over each level in your data structure so to be able to compare as needed

foreach my $ev (sort keys %data) { 
    my $ra = $data{$ev}; 
    
    my $fmt = "%Y-%m-%dT%T";

    # Initialize "best" time and its hashref
    my $hr = $ra->[0];
    my $dt = Time::Piece->strptime($hr->{Updated}, $fmt);
    
    foreach my $idx (1..$#$ra) { 
        my $dt_curr = Time::Piece->strptime(
            $ra->[$idx]{Updated}, $fmt
        );

        if ($dt_curr > $dt) {
            $dt = $dt_curr;
            $hr = $ra->[$idx];
        }
        elsif ($dt_curr == $dt) { 
            if ($ra->[$idx]{Severity} eq 'CLEARED') { 
                $dt = $dt_curr;
                $hr = $ra->[$idx];
            }
        }
    }   
    say "Event: $hr->{eventId}";
    say "\t", join ',', 
        map { $hr->{$_} } qw(Node Severity State Updated);
    # Or rather, as clarified in a comment
    #say join ',' 
    #    map { $hr->{$_} } qw(eventId Node Severity State Updated);
}    

With the provided %data, omitted for readability, this prints expected results.†

I assume that there aren't yet deeper levels in the data structure and so I merely copy hashrefs, otherwise they should be cloned (deep-copied).

The shown timestamps (%Y-%m-%dT%T) can be directly compared lexicographically as strings ($dt1 gt $dt2) but I still used a date-time module since it's then easy to adjust if a different timestamp format ends up being used.

The code can be tweaked for efficiency. (That also depends on data details.)


A question came up of how to do it without Time::Piece (or any library I take it), as it is unavailable in that install. Please see comments for what I'd like to say about working with such (crippled) tool, but here's a simple way. (This is also faster.)

sub parse_ts {               # add checks
    my ($timestamp) = @_;
    
    # For format year-mon-dayThour:minute:second
    return split /[-T:]/, $timestamp;
}

my ($yr, $mon, $day, $hr, $min, $sec) = parse_ts($timestamp);

The split's argument for the separator is a regex. As a char from the character class [-T:] is matched in the string (tried in the given order) the string is split on it. Their order is as in the timestamp so split returns: year, month, day, hour, minute, second.

With such a list for each timestamp we can intelligibly compare their date-times, by comparing components in that order (compare year, then month, ...).

However, with Time::Local (confirmed to be installed) you can avoid writing the tedious (and error prone) code to compare components, by obtaining seconds since epoch using timelocal and comparing that

use Time::Local;
# parse timestamps to get yr, mon, ... for each
my $dt1 = timelocal($sec1, $min1, $hour1, $mday1, $mon1, $yr1);
my $dt2 = ...

if ($dt1 > $dt2) { ... }

Altogether, with a rudimentary sanity check

sub ts_to_epoch {
    my ($timestamp) = @_;

    # For format year-mon-dayThour:minute:second
    my @dt_parts = split /[-T:]/, $timestamp;
    die "Parsing of '$timestamp' failed" if @dt_parts != 6;

    return timelocal( reverse @dt_parts );
}

my $epoch_1 = ts_to_epoch($timestamp1);
...

If you ever need to retrieve a timestamp from those seconds-since-epoch one way with your limited toolset is with localtime($epoch).


† The last line of code (commented out) prints in the desired order by listing keys in that order. But if that appears at many places in the code then we'd have hard-coded key-list scattered around; what when another key is added, or renamed? We'd have to hunt down all those places.

Instead, spell out the order in an ancillary hash

# One place in code where the hard-coded order is given
my %sort_by = do {
    my @keys = qw(eventId Node Severity State Updated);
    map { $keys[$_] => $_ } 0..$#keys;
};  

and now print by fetching the keys using keys and then sorting them

say join ',', 
    map { $hr->{$_} } 
    sort { $sort_by{$a} <=> $sort_by{$b} } 
    keys %$hr;

So %sort_by is given at one place in the code (and make it constant!), and printing statements don't use any specific, hard-coded data.

A small change in OP's code produces desired result

use strict;
use warnings;
use feature 'say';

use Data::Dumper;

my %data = (
          '1342' => [
                      {
                        'Severity' => 'MEDIUM',
                        'Node' => 'Node002',
                        'State' => 'ACTIVE_UNACKNOWLEDGED',
                        'Updated' => '2020-10-01T12:00:00',
                        'eventId' => '1342'
                      },
                      {
                        'Severity' => 'HIGH',
                        'Node' => 'Node002',
                        'Updated' => '2020-10-01T12:05:00',
                        'eventId' => '1342',
                        'State' => 'ACTIVE_UNACKNOWLEDGED'
                      }
                    ],
          '1341' => [
                      {
                        'State' => 'ACTIVE_UNACKNOWLEDGED',
                        'eventId' => '1341',
                        'Updated' => '2020-10-01T12:10:00',
                        'Node' => 'Node001',
                        'Severity' => 'HIGH'
                      },
                      {
                        'State' => 'ACTIVE_UNACKNOWLEDGED',
                        'Updated' => '2020-10-01T12:15:00',
                        'eventId' => '1341',
                        'Severity' => 'HIGH',
                        'Node' => 'Node001'
                      },
                      {
                        'State' => 'CLEARED_ACKNOWLEDGED',
                        'Updated' => '2020-10-01T12:15:00',
                        'eventId' => '1341',
                        'Severity' => 'CLEARED',
                        'Node' => 'Node001'
                      }
                    ]
);

foreach my $id ( sort keys %data ){
    print "Event: $id\n";
    my $found;
    for( @{$data{$id}} ) {
        $found = $_ unless defined $found;
        if( $_->{Updated} eq $found->{Updated} ) {
            $found = $_ if $_->{Severity} eq 'CLEARED';
        }
        $found = $_ if $_->{Updated} ge $found->{Updated};
    }
    say "\t" . join ',', $found->@{qw/Node Severity State Updated/};
}

Output

Event: 1341
        Node001,CLEARED,CLEARED_ACKNOWLEDGED,2020-10-01T12:15:00
Event: 1342
        Node002,HIGH,ACTIVE_UNACKNOWLEDGED,2020-10-01T12:05:00
Related