Using an OCR tool I extracted texts from screenshots (about 1-5 sentences each). However, when manually verifying the extracted text, I noticed several errors that occur from time to time.
Given the text "Hello there ! I really like Spark ❤️!", I noticed that:
1) Letters like "I", "!", and "l" get replaced by "|".
2) Emojis are not correctly extracted and replaced by other characters or are left out.
3) Blank spaces are removed from time to time.
As a result, I might end up with a string like this: "Hello there 7l | real|y like Spark!"
Since I am trying to match these string against a dataset including the correct text (in thise case "Hello there ! I really like Spark ❤️!"), I am looking for an efficient way how to match the string in Spark.
Can anyone suggest an efficient algorithm for Spark which allows me to compare the extract texts (~100.000) against my dataset (~100 million)?