I have been trying to remove the rows that contain alphabet characters as this one.
And what I tried is
df = df[~df['tag_name'].str.contains("[a-zA-Z]")]
I did remove some rows in this way but some rows remained like this. And I found these characters look different from those I type in.
归创通桥-ＢB
Could it be something wrong with the encoding method? And does anyone know how I can remove these rows?
Or you could try unicodedata.normalize:
import unicodedata
df = df[~df['tag_name'].apply(lambda x: unicodedata.normalize('NFKD', x)).str.contains("[a-zA-Z]")]
If you use print(u"B".encode('ascii','backslashreplace'))
you will see that the character is not actually a "B", but a FULLWIDTH LATIN CAPITAL LETTER B https://www.fileformat.info/info/unicode/char/ff22/index.htm
Which is why your regex cant detect it