Drop rows in pandas based on the same key

Viewed 141

How can I drop rows where column A is the key, and any rows for that key contain both "foo" and "moo" in column C

df_before:

"cat"   |"waverly way"|"foo"|10.0
"cat"   |"smokey st"  |"moo"|9.7
"rabbit"|"rapid ave"  |"foo"|6.6
"rabbit"|"far blvd"   |"too"|3.2

df_after:

"rabbit"|"rapid ave"  |"foo"|6.6
"rabbit"|"far blvd"   |"too"|3.2
3 Answers

You can do it this way:

df.columns = ['A', 'B', 'C', 'D']

df:

        A            B    C     D
0     cat  waverly way  foo  10.0
1     cat    smokey st  moo   9.7
2  rabbit    rapid ave  foo   6.6
3  rabbit     far blvd  too   3.2

.

filter = ['foo,moo']
# used transform to concat all the values in column 'C' for a given 'A'
df['C'] = df[['A', 'B', 'C', 'D']].groupby(by=['A'])['C'].transform(lambda x: ','.join(x))

Modified df:

        A            B        C     D
0     cat  waverly way  foo,moo  10.0
1     cat    smokey st  foo,moo   9.7
2  rabbit    rapid ave  foo,too   6.6
3  rabbit     far blvd  foo,too   3.2

.

# applied negative mask to filter values with foo,moo in 'C'
df1 = df.loc[~df['C'].isin(filter)].reset_index(drop=True)

print(df1)

Output:

        A          B        C    D
0  rabbit  rapid ave  foo,too  6.6
1  rabbit   far blvd  foo,too  3.2

I called each column A, B, and C. I called your df df_before.

Hope this helps (if it doesn't works completely :-P)

import pandas as pd
foomooList=['foo','moo']

df=df_before.copy()

dfA_unique_values= df.a.unique()
dfA_keepValues=[]

for x in dfA_unique_values:
    tempdf=df.copy()
    checkerDF=tempdf[tempdf.a==x]
    checkerDfC_unique_values = checkerDF.c.unique()

    if all(y in checkerDfC_unique_values for y in foomooList):
        pass #do nothing since both foo and moo are in the subset

    else:
        dfA_keepValues.append(x) #put A value in keep list

df_after=df_before[df_before.A.isin(dfA_keepValues)]

Here's another way of doing it:

x = df.groupby(by=['A'])['C'].agg(' '.join).str.contains((r'(?=.*foo)(?=.*moo)')).reset_index()
index = list(x[x['C']==True]['A'])
print(df[~df['A'].isin(index)])

        A            B    C    D
2  rabbit  rapid ave    foo  6.6
3  rabbit  far blvd     too  3.2
Related