I was trying to unify the lines in my file when I observed the following:
word1 word2
word1 word2
I did not understand why these lines were not combined so I opened the file in vim and used :set list to see if there are any special characters and I found this:
word1 <feff>word2
word1 word2
I am not sure how to clean this word in Python. Any suggestions on what character might be and how this can be cleaned?