Replacing method for words with boundaries in python (like with regex)

user_13427588 0
Viewed 189

I am seeking for a more robust replace method in python because I am building a spellchecker to input words in ocr-context.

Let's say we have the following text in python:

text =  """
this is a text, generated using optical character recognition. 
this ls having a lot of errors because
the scanned pdf has too bad resolution.
Unfortunately, his text is very difficult to work with. 
"""

It is easy to realize that instead of "his is a text" the right phrase would be "this is a text". And if I do text.replace('his','this') then I replace every single 'his' for this, so I would get errors like "tthis" is a text. When I do a replacement. I would like to replace the whole word 'this' and not his or this. Why not trying this?

word_to_replace='his'
corrected_word = 'this'
corrected_text = re.sub('\b'+word_to_replace+'\b',corrected_word,text)
corrected_text

Awesome, we did it, but the problem is... what if the word to correct contains an special character like '|'. For example, '|ights are on' instead of 'lights are one'. Trust me, it happened to me, the re.sub is a disaster in that case. The question is, have you encountered the same problem? Is there any method to solve this? The replacement is the most robust option. I tried text.replace(' '+word_to_replace+' ',' '+word_to_replace+' ') and this solve a lot of things but still have the problem of phrases like "his is a text " because the replacement doesnt work here since 'his' is at the begining of a sentence and not ' his ' for ' this '.

Is there any replacement method in python that takes the whole word like in regexs \b word_to_correct \b as input ?

nlp python replace regex
1 Answers

after a few days I solved the problem that I had. I hope this could be helpful for someone else. Let me know if you have any question or something. 


text =  """
this is a text, generated using optical character recognition. 
this ls having a lot of errors because
the scanned pdf has too bad resolution.
Unfortunately, his text is very difficult to work with. 
"""


# Asume you already have corrected your word via ocr 
# and you just have to replace it in the text (I did it with my ocr spellchecker)
# So we get the following word2correct and corrected_word (word after spellchecking system)
word2correct = 'his'
corrected_word = 'this'

#
# now we replace the word and the its context
def context_replace(old_word,new_word,text):
    # Match word between boundaries \\b\ using regex. This will capture his and its context but not this  and its context
    phrase2correct = re.findall('.{1,10}'+'\\b'+word2correct+'\\b'+'.{1,10}',text)[0]
    # Once you matched the context, input the new word 
    phrase_corrected = phrase2correct.replace(word2correct,corrected_word)
    # Now replace  the old phrase (phrase2correct) with the new one *phrase_corrected
    text = text.replace(phrase2correct,phrase_corrected)
    return text

Test if the function works...

print(context_replace(old_word=word2correct,new_word=corrected_word,text=text))

Output:

this is a text, generated using optical character recognition. 
this ls having a lot of errors because
the scanned pdf has too bad resolution.
Unfortunately, this text is very difficult to work with.

It worked for my purpose. I hope this is helpful for someone else.

user_134275880
Related