Filtering data from elastic search based on mongodb

user_4213094 0
Viewed 225

I've a list of items in my ElasticSearch. User enters a query and I fetch the results from elastic search. Now, I've some user preferences stored in mongodb based on which I want to filter the results of elastic search.

  1. Suppose, I get a list of items(item_ids) from Elasticsearch.

  2. Mongo DB has following schema. id, user_id, item_id

  3. I choose this MongoDB schema because a user could have a very big list of items(in order of Millions), which he doesn't want to see in results.

How do I achieve this with scale? Do I need to change my schema?

elasticsearch mongodb database-design
1 Answers

You should use elasticsearch filtering for this, you can include the filter criteria in your ES query which would reduce the number of results to return without which

  1. You have to return huge data set from ES and then do the filtering in MongoDB which is two step process and costly at both ES and mongo side.
  2. With filters at ES, it would return less data which would avoid extra post-processing at mongoDB and filters are executed first and by default cached at elasticsearch side so you don't need further caching solution like redis etc.

Refer filter and query context and from same official doc, info about filter cache.

Frequently used filters will be cached automatically by Elasticsearch, to speed up performance.

user_40394310
Related