I want to calculate the count of source fields that is from the last document in every other group.
Let's take an example here, Suppose I have the following document in the ES
{"name": "abc", "external_key": "X21W", "created_at": "2020-11-03T11:22:17.213000"}
{"name": "xyz", "external_key": "X21W", "created_at": "2020-11-03T10:22:17.213000"}
{"name": "pqr", "external_key": "X21W", "created_at": "2020-11-02T09:22:17.213000"}
{"name": "abc", "external_key": "I12K", "created_at": "2020-11-04T08:22:17.213000"}
{"name": "pqr", "external_key": "I12K", "created_at": "2020-11-03T01:22:17.213000"}
{"name": "xyz", "external_key": "AB23", "created_at": "2020-11-03T02:22:17.213000"}
{"name": "pqr", "external_key": "AB23", "created_at": "2020-11-03T08:22:17.213000"}
Now, Query should create a group based on external_key and find the one name from a document which has max created_at
Here we have three groups, from that the following documents will be selected.
{"name": "abc", "external_key": "X21W", "created_at": "2020-11-03T11:22:17.213000"}
{"name": "abc", "external_key": "I12K", "created_at": "2020-11-04T08:22:17.213000"}
{"name": "pqr", "external_key": "AB23", "created_at": "2020-11-03T08:22:17.213000"}
Here document with the name abc is selected 2 times and pqr is selected 1 time.
I have created a query to fetch the last document from each group,
{
"size": 0,
"aggs": {
"top_documents": {
"terms": {
"field": "external_key",
"size": 1000
},
"aggs": {
"last_name": {
"top_hits": {
"_source": [
"name"
],
"size": 1,
"sort": [
{
"created_at": {
"order": "desc"
}
}
]
}
}
}
}
}
}
So, Query must return abc= 2 and pqr= 1.
How can we achieve that in the above Query?