I am tasked with matching free form text to data in a database. What I mean by freeform, is that it is a textbox, and someone can type something/anything. For the most part, these entries are valid. I would like to find a list of values from a table that resemble what was typed in. Before you ask, I have no control of said textbox, nor the people that type into it. I am looking for techniques, not specific technologies.
Things I have tried:
- Clearing out the common words from both the criteria as well as the list. ie (the, of, in, etc.)
- SOUNDEX function in sql, it is very weak, and not quite helpfull.
- The Levenshtein Distance algorithm and am pretty happy with the results, but it still needs lots of polish.
For example I have this list:
- The Hobbit: An Unexpected Journey
- The Hobbit: The Desolation of Smaug
- The Hobbit: There and Back Again
- Iron Man 3
- Despicable Me 2
- Fast & Furious 6
- Monsters University
- The Hunger Games: Catching Fire
- Man of Steel
- Gravity
- Thor: The Dark World
- The Croods
- World War Z
The users input could be:
- hobit unexpected journ
- The word 'hobit' is not spelled right
- Expected result:
- The Hobbit: An Unexpected Journey
- The Hobbit: There and Back Again
- The Hobbit: The Desolation of Smaug
- hunger game
- Expected result:
- The Hunger Games: Catching Fire
- Expected result:
What I guess I'm asking is what other methods can I use to calculate these results. My Stack is .Net 4.0 and MSSQL 2008 R2