Is it possible for Lucene to monitor a Sql Table and keep itself updated?

user_1794286 0
Viewed 156

I am trying to understand some basics of Lucene, the full text search engine. More specifically I am looking at Lucene.Net.

Today I have an old legacy .NET 4.8 web app. Some is MVC, but the newer parts follow a pretty nice API first pattern. The app holds a lot of records (app half a million) with tons of different fields. The search functionality there is outdated to say the least. It is a ton of old Linq2SQL queries that fan out in like queries.

I would like to introduce a new and better way to search records, so I started looking at Lucene.Net. But I am trying to understand one key concept, and I can't seem to find the answer anywhere, and I think it might be because it cannot be done, but I would like to make sure.

Is it possible to set up Lucene to monitor a SQL table or view so I don't have to maintain the Lucene index from within my code. The code of this app does not lend itself to easily keeping a Lucene index updated when things are added, changed or deleted. But the database is good source of truth. I can live with a small delay on having the index up to date. But basically I would like define for each business model what fields are part of the index and what the id is, and then be able to query with that index from the C# server side code of my Web App.

Is such a scenario even possible or am I asking too much?

lucene.net lucene full-text-search c#
1 Answers

It's totally possible, but not out of the box. You have to implement it if you want it. Fundamentally you need to implement three things.

  1. A way to know every time a piece of relevant data in the sql database changes
  2. A place to capture information about that change, call it a change log.
  3. A routine that reads the change log, applies those changes to the

LuceneNet index and than marks the record in the change log has processed.

There are of course lots of different ways to handle each of these.

This SO answer Lucene.Net index updates, when a manual change is done in SQL Database provides more details on one way this can be accomplished.

user_14156140
Related