Add new fields in 10 million documents in ElasticSearch

user_6878839 0
Viewed 2607

I have ElasticSearch index that stores more than 10 mln documents. I need to iterate over all index and add new field and value for each document. I know about two approaches: 1. Use scroll search to get all results and use BULK api to copy all documents into new index [adding new fields while copying]. After all documents are copied to new index - switch aliases. 2. Use scroll search and bulk API to update all documents in existed index (do not need to copy to new index and switching aliases).

I do not understand, why first approach is recommended? Can I just use 2 approach? Is there any possibility that while running program with 2 approach, something bad can happen to the index? Index that I need to update has live prod data and I worried that customers may be affected in some way

elasticsearch
1 Answers

Use the script painless 

POST /mysqltest/_update_by_query
{
  "query": {
    "match_all": {}
  },
 "script": { "inline": "ctx._source.category = \"10\";" } --> category is new key that to be added in all documents
}
user_44152310
Related