How to use AND condition in pandas when dealing with value_counts?

user_17601386 0
Viewed 29

Basically, I want to filter a Pandas DataFrame by the value_counts(), however, I also want to make an exception. For example, this is a sample code:

import pandas as pd

df = pd.DataFrame({
    'city': ['NYC', 'NYC', 'SYD', 'NYC', 'SEL', 'NYC', 'NYC']
})

value_counts = df['city'].value_counts()
to_remove = value_counts[value_counts <= 3].index

df = df[(~df.city.isin(to_remove)) & (df.city == 'SEL')]

I would like to display both the NYC, which meets the value_counts() condition, but I also want to make an exception for 'SEL'

pandas dataframe python
1 Answers

Use | for bitwise OR:

df = df[(~df.city.isin(to_remove)) | (df.city == 'SEL')]
print (df)
  city
0  NYC
1  NYC
3  NYC
4  SEL
5  NYC
6  NYC

Another solution with Series.map and Series.gt for invert mask from <= to >:

value_counts = df['city'].value_counts()

df = df[df['city'].map(value_counts).gt(3) | (df.city == 'SEL')]
print (df)
  city
0  NYC
1  NYC
3  NYC
4  SEL
5  NYC
6  NYC
user_29010020
Related