Return the most recent record from ElasticSearch index

Viewed 127330

I would like to return the most recent record (top 1) from ElasticSearch index similar to the sql query below;

SELECT TOP 1 Id, name, title 
FROM MyTable 
ORDER BY Date DESC;

Can this be done?

8 Answers

If you are using python elasticsearch5 module or curl:

  1. make sure each document that gets inserted has
    • a timestamp field that is type datetime
    • and you are monotonically increasing the timestamp value for each document
  2. from python you do

    es = elasticsearch5.Elasticsearch('my_host:my_port')
    es.search(
        index='my_index', 
        size=1,
        sort='my_timestamp:desc'
        )
    

If your documents are not inserted with any field that is of type datetime, then I don't believe you can get the N "most recent".

Since this question was originally asked and answered, some of the inner-workings of Elasticsearch have changed, particularly around timestamps. Here is a full example showing how to query for single latest record. Tested on ES 6/7.

1) Tell Elasticsearch to treat timestamp field as the timestamp

curl -XPUT "localhost:9200/my_index?pretty" -H 'Content-Type: application/json' -d '{"mappings":{"message":{"properties":{"timestamp":{"type":"date"}}}}}'

2) Put some test data into the index

curl -XPOST "localhost:9200/my_index/message/1" -H 'Content-Type: application/json' -d '{ "timestamp" : "2019-08-02T03:00:00Z", "message" : "hello world" }'
curl -XPOST "localhost:9200/my_index/message/2" -H 'Content-Type: application/json' -d '{ "timestamp" : "2019-08-02T04:00:00Z", "message" : "bye world" }'

3) Query for the latest record

curl -X POST "localhost:9200/my_index/_search" -H 'Content-Type: application/json' -d '{"query": {"match_all": {}},"size": 1,"sort": [{"timestamp": {"order": "desc"}}]}'

4) Expected results

{
   "took":0,
   "timed_out":false,
   "_shards":{
      "total":5,
      "successful":5,
      "skipped":0,
      "failed":0
   },
   "hits":{
      "total":2,
      "max_score":null,
      "hits":[
         {
            "_index":"my_index",
            "_type":"message",
            "_id":"2",
            "_score":null,
            "_source":{
               "timestamp":"2019-08-02T04:00:00Z",
               "message":"bye world"
            },
            "sort":[
               1564718400000
            ]
         }
      ]
   }
}

I used @timestamp instead of _timestamp

{
    'size' : 1,
    'query': {
        'match_all' : {}
            },
    "sort" : [{"@timestamp":{"order": "desc"}}]
}
Related