How to use multiple values in a single np.where condition?

user_10829044 0
Viewed 180

I have a dataframe like as shown below

df = pd.DataFrame({'text': ["Hi how","I am fine","Ila say Hi","hello"],
                   'tokens':["test","correct","Tim",np.nan],
                   'labels':['A','B','C','D']})

Instead of multiple np.where conditions, I would like to use Or or | operator to check for multiple values in np.where condition like below

df['labels'] = np.where(df['tokens'] == ('test'|'correct'|is.na()),'new_label',df['labels'])

However, this results in an error

TypeError: unsupported operand type(s) for |: 'str' and 'str'

I expect my output to be like as shown below. How can I do this efficiently for big data with million of records?

enter image description here

pandas series dataframe numpy python
1 Answers

First idea is replace missing values by some value from list, e.g. test and then compare by Series.isin:

df['labels'] = np.where(df['tokens'].fillna('test').isin(['test','correct']),
                        'new_label',
                        df['labels'])
print (df)
         text   tokens     labels
0      Hi how     test  new_label
1   I am fine  correct  new_label
2  Ila say Hi      Tim          C
3       hello      NaN  new_label

Or chain another mask by | for bitwise OR form compare NaNs:

df['labels'] = np.where(df['tokens'].isin(['test','correct']) | df['tokens'].isna(),
                        'new_label',
                        df['labels'])
print (df)
         text   tokens     labels
0      Hi how     test  new_label
1   I am fine  correct  new_label
2  Ila say Hi      Tim          C
3       hello      NaN  new_label
user_29010020
Related