fuse.js takes 10+ seconds with semi-long queries

user_12359120 0
Viewed 510

I have a JSON database with 27k+ entries, and I'm using fuse.js to search through them. Each entry has about 500 characters of text in it.

When I search for something with 15+ characters, it takes a few seconds, and even more can slow the server to a halt while it processes it.

e.g. 1 results for 'crunchy munchy cheeeese' found in about 3.40 seconds.

var search = new Fuse(db.sites, {
  keys: ['t', 'dc','kw'], // Title, description and keywords
  threshold:0.4,
  minMatchCharLength:3
})
setInterval(() => {
  search.setCollection(db.sites) // Update the documents to the latest ones
}, 120000);

Any help on how to make this faster?

fuse.js node.js performance
1 Answers

I've run into issues w/Fuse being really slow with large datasets. Unfortunately the best solution for me was to just implement a basic fuzzy search myself by converting each searched property value to lowercase and comparing that against the search query which I also lowercased (to avoid case-sensitive issues) and then run it in a separate thread so that it doesn't block the main thread. Fuse doesn't seem to be designed for large datasets.

user_19165010
Related