I have messages which are like below, the following message is one of the messages (have so many JSON formats which are not at all related to this)
request body to the server {'sender': '65ddd20eac244AAe619383e4d8cb558834', 'message': 'hello'}
I would like to group of these messages based on sender (alphanumeric value) which is enclosed in JSON.
CloudWatch Logs Insights query:
fields @message |
filter @message like 'request body to the server' |
parse @message "'sender': '*', 'message'" as sender |
stats count(*) by sender
Query results:
-------------------------------------------------
| sender | count(*) |
|------------------------------------|----------|
| 65ddd20eac244AAe619383e4d8cb558834 | 4 |
| 55ddd20eac244AAe619383e4d8cb558834 | 3 |
-------------------------------------------------
Screenshot:
you can use filter.
fields @timestamp, @message
| filter @message like "65ddd20eac244AAe619383e4d8cb558834"
| sort @timestamp desc
| limit 20
it will filter all the messages limit to 20 that send by 65ddd20eac244AAe619383e4d8cb558834.
update:
suppose the JSON log formate is this
{
"sender": "65ddd20eac244AAe619383e4d8cb558835",
"message": "Hi"
}
Now I want to count number of messages from 65ddd20eac244AAe619383e4d8cb558835
how many messages are coming from each user?
so simple you can run the query
stats count(sender) by sender |
# To filter only message the contain sender, to avoid lambda default logs
filter @message like "sender"
if you want to see messages as well then modify the query a bit
stats count(*) by sender, message |
filter @message like "sender"
Here @message refers to whole to index where message refer to the JSON object message.
count_distinct
Returns the number of unique values for the field. If the field has very high cardinality (contains many unique values), the value returned by
count_distinctis just an approximation.
how many distinct users in the selected interval?
It will list distinct users in 3hr of interval
stats count_distinct(sender) as distinct_sender by bin(3hr) as interval
