Replace specific chinese punctuations with correspondent english ones in Python

user_8410477 0
Viewed 46

Given a test dataset as follows:

   id                   company
0   1                  xyz，ltd。
1   2  wall street english （bj）
2   3                 James（sh）
3   4                       NaN
4   5                    黑石（上海）

I need to replace chinese punctutations with correspondent english one: ( for （, ) for ）, . for 。 and , for ，.

I try with dd.company.str.replace('（', '(').replace('）', ')').replace('。', '.').replace('，', ','), it's not pythonic solution and not work out either.

Out:

0                    xyz，ltd。
1    wall street english (bj）
2                   James(sh）
3                         NaN
4                      黑石(上海）
Name: company, dtype: object

How could I replace them correctly? Thanks a lot.

pandas dataframe python-3.x regex string
2 Answers

One idea is use 2 lists or dictionary and pass regex=True for substring replacement:

dd.company.replace(['（','）', '。', '，'], ['(',')','.', ','], regex=True)

dd.company.replace({'（':'）', '(':')', '。':'.', '，':','}, regex=True)
user_29010020

I recommend matching any punctuation (say, with [^\w\s] regex that matches any non-word and non-whitespace chars) and apply the unidecode on the match:

import pandas as pd
import numpy as np
import unidecode
df = pd.DataFrame({'id':[1,2,3,4,5], 'company': ['xyz，ltd。', 'wall street english （bj）', 'James（sh）', np.NaN, '黑石（上海）']})
df['company'].str.replace(r'[^\w\s]+', lambda x: unidecode.unidecode(x.group()))
>>> df['company'].str.replace(r'[^\w\s]+', lambda x: unidecode.unidecode(x.group()))
0                   xyz,ltd. 
1    wall street english (bj)
2                   James(sh)
3                         NaN
4                      黑石(上海)
Name: company, dtype: object
user_38329700
Related