Does elasticsearch's bulk api can ensure that the docs are written into es ordered?

Viewed 565

I have checked the official doc and the doc syas:

The bulk API’s response contains the individual results of each operation in the request, returned in the order submitted. The success or failure of an individual operation does not affect other operations in the request.

The official document doesn't tell whether the doc are written into elasticsearch with order if they have the same primary key '_id'.

Here have a example. I have 3 docs with the same pk id=1, and other field is different. And I create 3 UpdateRequest for these 3 docs. I add these docs to BulkRequest with order. After submit to es client, can I ensure these docs execute in es with the order I add them to bulk request?

// Three docs with same id but different non-pk field value 
Map<String,String> doc1 = new HashMap;
doc1.put("_id","1");
doc1.put("other_column","columnValue1");

Map<String,String> doc2 = new HashMap;
doc1.put("_id","1");
doc1.put("other_column","columnValue2");

Map<String,String> doc3 = new HashMap;
doc1.put("_id","1");
doc1.put("other_column","columnValue3");


//prepare 3 update Request
 UpdateRequest updateRequest1 = new UpdateRequest(index,type,"1").docAsUpsert(true).setDoc(doc1);
 UpdateRequest updateRequest2 = new UpdateRequest(index,type,"1").docAsUpsert(true).setDoc(doc2);
 UpdateRequest updateRequest3 = new UpdateRequest(index,type,"1").docAsUpsert(true).setDoc(doc3);



//add the to bulk request
BulkRequest bulkRequest = new BulkRequest();
bulkRequest.add(indexRequest1);
bulkRequest.add(indexRequest2);
bulkRequest.add(indexRequest3);

//execute by client
restHighLevelClient.bulk(bulkRequest);

//if doc2 write into elasticsearch failed, doc1 and doc2 may also be executed successfully as the official doc said. Is it right?



However,the doc says that the success or failure of an individual operation does not affect other operations in the request. In my case, if the docs in the es request with same pk have part of failure docs and the other docs write into es successfully. Abviously,it means they are not write into es with order. Does the conclusion that es client bulk api can't ensure doc write into es with order is right?

1 Answers

If you are inserting records using only bulk processor ,then during flushing the records are flushed in ordered manner, according to timestamp. So the document will be updated with the latest data for the particular doc id. Lets say if you index a doc and then delete it. So the final result will be doc being deleted.

But if there are normal indexing going on other than bulk, and if the same id is indexed using both bulk and regular indexing api, then the ordering is not maintained. As the flushing may occur in different time. Lets say you indexed a doc with id 1 using bulk, then you deleted the id 1 using regular indexing api. If the flush interval is set 5 seconds for bulk, then it may happen that your deleted index will remain indexed in es.

Related