Elasticsearch: aggregation on result source generated by another aggregation

Viewed 43

I want to calculate the count of source fields that is from the last document in every other group.

Let's take an example here, Suppose I have the following document in the ES

{"name": "abc", "external_key": "X21W", "created_at": "2020-11-03T11:22:17.213000"}
{"name": "xyz", "external_key": "X21W", "created_at": "2020-11-03T10:22:17.213000"}
{"name": "pqr", "external_key": "X21W", "created_at": "2020-11-02T09:22:17.213000"}

{"name": "abc", "external_key": "I12K", "created_at": "2020-11-04T08:22:17.213000"}
{"name": "pqr", "external_key": "I12K", "created_at": "2020-11-03T01:22:17.213000"}

{"name": "xyz", "external_key": "AB23", "created_at": "2020-11-03T02:22:17.213000"}
{"name": "pqr", "external_key": "AB23", "created_at": "2020-11-03T08:22:17.213000"}

Now, Query should create a group based on external_key and find the one name from a document which has max created_at

Here we have three groups, from that the following documents will be selected.

{"name": "abc", "external_key": "X21W", "created_at": "2020-11-03T11:22:17.213000"}
{"name": "abc", "external_key": "I12K", "created_at": "2020-11-04T08:22:17.213000"}
{"name": "pqr", "external_key": "AB23", "created_at": "2020-11-03T08:22:17.213000"}

Here document with the name abc is selected 2 times and pqr is selected 1 time.

I have created a query to fetch the last document from each group,

{
    "size": 0,
    "aggs": {
        "top_documents": {
            "terms": {
                "field": "external_key",
                "size": 1000
            },
            "aggs": {
                "last_name": {
                    "top_hits": {
                        "_source": [
                            "name"
                        ],
                        "size": 1,
                        "sort": [
                            {
                                "created_at": {
                                    "order": "desc"
                                }
                            }
                        ]
                    }
                }
            }
        }
    }
}

So, Query must return abc= 2 and pqr= 1.

How can we achieve that in the above Query?

0 Answers
Related