How to get complete data from ElasticSearch python for large dataset. Search with match all returns by default 10 and max 10000

Viewed 187

self.conn.Elasticsearch(index = index_name, body={"query": {"match_all": {}}}, size = 10000) this works fine for now but in the case when there are million points in the ElasticSearch this would fail. I have even tried scrolloperation on ElasticSearch containing 136 records but got result only 36. Here is the code snippet :

resp = self.conn.Elasticsearch(index = index_name, body={"query": {"match_all": {}}}, size = 1000, scroll = '10s')
scroll_id = resp['_scroll_id']
resp = client.scroll(
        scroll_id = scroll_id,
        scroll = '5s')

this only provides 36 response. Please provide info on how i can get maximum size at once or stream the data.

0 Answers
Related