how could I achieve something like np.where(df[varaible] in ['value1','value2'])

user_8350907 0
Viewed 61

Hi I want to change one categorical variable's value to other in the condition like ['value1','value2']

Here is my code:

random_sample['NAME_INCOME_TYPE_ind'] = np.where(random_sample['NAME_INCOME_TYPE'] in ['Maternity leave', 'Student']), 'Other')

I tried adding .any() in different position of this line of code, but it still does not resolve the error. ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all().

pandas categorical-data series numpy python
2 Answers

Use Categorical Data for categorical variables

When dealing with categoricals, you can replace categories with another rather than replacing strings. This has memory and performance benefits, as internally Pandas uses factorisation for categorical data.

df = pd.DataFrame({'NAME_INCOME_TYPE': ['Employed', 'Maternity leave',
                                        'Benefits', 'Student']})

# turn object series to categorical
label_col = 'NAME_INCOME_TYPE'
df[label_col] = df[label_col].astype('category')

# define others
others = ['Maternity leave', 'Student']
others_label = 'Other'

# add new category and replace existing categories
df[label_col] = df[label_col].cat.add_categories([others_label])
df[label_col] = df[label_col].replace(others, others_label)

print(df)

  NAME_INCOME_TYPE
0         Employed
1            Other
2         Benefits
3            Other

You can also write this more succinctly using method chaining:

# define others
others, others_label = ['Maternity leave', 'Student'], 'Other'

# turn to categorical, add category, then replace
df['NAME_INCOME_TYPE'] = df['NAME_INCOME_TYPE'].astype('category')\
                                               .cat.add_categories([others_label])\
                                               .replace(others, others_label)
user_92095460

You can use str.contains to check where the condition is satisfied:

l = ('|').join(['Maternity leave', 'Student'])
m = random_sample['NAME_INCOME_TYPE'].str.contains(l)

You could also generate m using .isin:

random_sample['NAME_INCOME_TYPE'].isin(['Maternity leave', 'Student'])

And then use np.where. However, note that you cannot only specify one of the two values from which to choose depending on the condition, you have to specify both x and y. For your case you could use df['NAME_INCOME_TYPE'] and other as x and y:

random_sample['NAME_INCOME_TYPE_ind'] = np.where(m, 
                                                'Other',
                                                random_sample['NAME_INCOME_TYPE'])

Test on an example dataframe:

df = pd.DataFrame({'NAME_INCOME_TYPE':['word1','word2','Student']})

l = ('|').join(['Maternity leave', 'Student'])
m = random_sample['NAME_INCOME_TYPE'].str.contains(l)
df['NAME_INCOME_TYPE_ind'] = np.where(m, 'Other', df['NAME_INCOME_TYPE'])

       NAME_INCOME_TYPE NAME_INCOME_TYPE_ind
0            word1                word1
1            word2                word2
2          Student                Other
user_96986840
Related