What should we do exactly during the preprocessing step for GPT2? Are there any guidelines?
Would this be just fine for the preprocessing step?
1. Remove any \n from sentence
2. Remove extra spaces from sentence
3. Leave everything else that is part of the sentence but not exactly words (e.g. urls, non-english words that may be added in an english sentence, emojis, etc...)
Wouldn't it be better to remove extra punctuation or any non-english character?