SumoLogic — Plotting data from a "status" json message in the log

user_1614242 0
Viewed 332

I have a service that accepts and processes tasks. A Task has a status: queued, running, failed, cancelled or finished. Once in a while the service spits out a log entry with the json, like this:

2021-09-09 00:30:46,742 [Timer-0] INFO - { "env": "test_environment", "capacity": 10, "available_ec2": 10, "failed_ec2": 0, "running_tasks": 0, "queued_tasks": 0, "finished_tasks": 0, "failed_tasks": 0, "cancelled_tasks": 3,"queue_wait_minutes" : { "max": 0, "mean": -318990, "max_started": 0, "mean_started": -29715 },"processing_time": {"max": 0, "mean": 0} }

I would like to plot a pie chart that would show the breakdown of the tasks by status ("running_tasks", "queued_tasks", "finished_tasks", "failed_tasks":, "cancelled_tasks" in the json message). So far I have failed to do so, because I cannot come up with how to construct a table out of such message. Any clues would be highly appreciated — thanks in advance!

sumologic
3 Answers

Try something like this. Basically, you have to de-transpose the data. I hope this makes sense!

...
| parse field=some_log_line "INFO - *" as jsonMessage
| json field=jsonMessage "running_tasks"
| json field=jsonMessage "queued_tasks"
| json field=jsonMessage "finished_tasks"
| "running_tasks,queued_tasks,finished_tasks," as message_keys
| parse regex field=message_keys "(?<message_key>.*?)," multi
| if (message_key="running_tasks", running_tasks, 0) as message_value
| if (message_key="queued_tasks", queued_tasks, message_value) as message_value
| if (message_key="finished_tasks", finished_tasks, message_value) as message_value
| fields message_key, message_value
| max(message_value) by message_key
user_13057180

First of all, Sumo Logic supports parsing JSON into fields. In your example not the whole line is a JSON, but only the part after "-", so you can add this to your query:

...
| parse "INFO - *" as jsonMessage
| json auto

Then, you can use running_tasks, queued_tasks, etc. as ordinary fields, e.g.

...
| timeslice 1m
| max(running_tasks), max(queued_tasks) by _timeslice

Disclaimer: I am currently employed by Sumo Logic.

user_1185870

Below is a pure python solution that will you plot the data.

The output (entries) is a dict where the key is the time stamp and the value is a dict that contains the interesting info. log_lines holds a collection of log messages and is used as the input.

import json
import pprint

log_lines = [
    '2021-09-09 00:30:46,742 [Timer-0] INFO - { "env": "test_environment", "capacity": 10, "available_ec2": 10, "failed_ec2": 0, "running_tasks": 2, "queued_tasks": 0, "finished_tasks": 0, "failed_tasks": 0, "cancelled_tasks": 3,"queue_wait_minutes" : { "max": 0, "mean": -318990, "max_started": 0, "mean_started": -29715 },"processing_time": {"max": 0, "mean": 0} }',
    '2021-09-09 00:31:46,742 [Timer-0] INFO - { "env": "test_environment", "capacity": 10, "available_ec2": 10, "failed_ec2": 0, "running_tasks": 5, "queued_tasks": 0, "finished_tasks": 0, "failed_tasks": 0, "cancelled_tasks": 3,"queue_wait_minutes" : { "max": 0, "mean": -318990, "max_started": 0, "mean_started": -29715 },"processing_time": {"max": 0, "mean": 0} }'
]
entries = dict()

for line in log_lines:
    date = line[:line.find('[') - 1]
    data = json.loads(line[line.find('{'):])
    sub_set = {k: data.get(k,0) for k in
               ["running_tasks", "queued_tasks", "finished_tasks", "failed_tasks", "cancelled_tasks"]}
    entries[date] = sub_set
pprint.pprint(entries)

output

{'2021-09-09 00:30:46,742': {'cancelled_tasks': 3,
                             'failed_tasks': 0,
                             'finished_tasks': 0,
                             'queued_tasks': 0,
                             'running_tasks': 2},
 '2021-09-09 00:31:46,742': {'cancelled_tasks': 3,
                             'failed_tasks': 0,
                             'finished_tasks': 0,
                             'queued_tasks': 0,
                             'running_tasks': 5}}
user_4150160
Related