Good afternoon. Checking the text in the column, I came across characters that I didn't need:
"|,|.|
|2|5|0|1|6|ё|–|8|3|-|c|t|r|l|+|e|n|g|i|w|s|k|z|«|(|)|»|—|9|7|?|o|b|a|/|f|v|:|%|4|!|;|h|y|u|d|&|j|p|x|m|і|№|ұ|…|қ|$|_|[|]|“|”|ғ||||>|−|„|*|¬|ү|ң|@|©|―|q|→|’|∙|·| |ә| |ө|š|é|=||×|″|⇑|⇐|⇒|‑|′|\|<|#|'|˚| |ü|̇|̆|•|½|¾|ń|¤|һ|ý|{|}| |‘|ā|í||ī||ќ|ђ|°|‚|ѓ|џ|ļ|▶|新|千|歳|空|港|全|日|機|が|曲|り|き|れ|ず|に|雪|突|っ|込|む|ニ|ュ|ー|ス|¼|ù|~|ə|ў|ҳ|ό||€||¸|⠀|ä|¯|ツ|ї|ş|è|`|́|ҹ|®|²||ç| |☑|️|‼|ú|‒|||||ó|±|ñ|ł|ش|ا|ه|ن|م|›|
|£||||º
Text encoding - UTF8.
How do I correctly remove all these characters from a specific column (series) of a Pandas data frame?
I try
template = bad_symbols[0].str.cat(sep='|')
print(template)
template = re.compile(template, re.UNICODE)
test = label_data['text'].str.replace(template, '', regex=True)
And I get the following error:
"|,|.|
|2|5|0|1|6|ё|–|8|3|-|c|t|r|l|+|e|n|g|i|w|s|k|z|«|(|)|»|—|9|7|?|o|b|a|/|f|v|:|%|4|!|;|h|y|u|d|&|j|p|x|m|і|№|ұ|…|қ|$|_|[|]|“|”|ғ||||>|−|„|*|¬|ү|ң|@|©|―|q|→|’|∙|·| |ә| |ө|š|é|=||×|″|⇑|⇐|⇒|‑|′|\|<|#|'|˚| |ü|̇|̆|•|½|¾|ń|¤|һ|ý|{|}| |‘|ā|í||ī||ќ|ђ|°|‚|ѓ|џ|ļ|▶|新|千|歳|空|港|全|日|機|が|曲|り|き|れ|ず|に|雪|突|っ|込|む|ニ|ュ|ー|ス|¼|ù|~|ə|ў|ҳ|ό||€||¸|⠀|ä|¯|ツ|ї|ş|è|`|́|ҹ|®|²||ç| |☑|️|‼|ú|‒|||||ó|±|ñ|ł|ش|ا|ه|ن|م|›|
|£||||º
---------------------------------------------------------------------------
error Traceback (most recent call last)
<ipython-input-105-36817f343a8a> in <module>
5 print(template)
6
----> 7 template = re.compile(template, re.UNICODE)
8
9 test = label_data['text'].str.replace(template, '', regex=True)
5 frames
/usr/lib/python3.7/sre_parse.py in _parse(source, state, verbose, nested, first)
643 if not item or item[0][0] is AT:
644 raise source.error("nothing to repeat",
--> 645 source.tell() - here + len(this))
646 if item[0][0] in _REPEATCODES:
647 raise source.error("multiple repeat",
error: nothing to repeat at position 36 (line 2, column 30)