Filtering with conditional conditions (pandas)

user_16352074 0
Viewed 49

I have an array with multiple strings, some of them are none(0 or ''), and each of them should have their own conditions if exists. if the array at its position is none, I don't have to apply the filtering.

# df.columns = ['a','b','c','d','e']

# Case 1 
l = ['A', 'B', '','' , 123]
## DESIRED FILTERING
df[ (df.a=='A') & (df.b=='B') & (df.e == 123)]

# Case 2
l = ['z','' ,'' ,'', 123]
## DESIRED FILTERING
df[ (df.a=='z') & (df.e == 123) ]

This is my attempt, yet it failed cuz (df.col_name == 'something') returns a series.

#Case 1 for example

check_null = [ i!='' for i in l ]       # ->returns [true,false,...]
conditions = [ (df.a==l[0]),(df.b==l[1]),(df.c==l[2]), (df.d==l[3]), (df.e==l[4])]
filt = [conditions[i] for i in range(len(check_null)) if check_null[i]]

df[filt]

How do I manage to get this work?

pandas filtering dataframe python
2 Answers

Create dictionary for non empty values, convert to Series and filtering in boolean indexing:

df = pd.DataFrame(columns = ['a','b','c','d','e'])
df.loc[0] = ['A', 'B','g' ,'h' , 123]
df.loc[1] = ['A', 'B','g' ,'h' , 52]

l = ['A', 'B','' ,'' , 123]

s = pd.Series(dict(zip(df.columns, l))).loc[lambda x: x != '']

df  = df[df[s.index].eq(s).all(axis=1)]
print (df)
   a  b  c  d    e
0  A  B  g  h  123

l = ['A', 'B', '','', '']
s = pd.Series(dict(zip(df.columns, l))).loc[lambda x: x != '']

df  = df[df[s.index].eq(s).all(axis=1)]
print (df)
   a  b  c  d    e
0  A  B  g  h  123
1  A  B  g  h   52
user_29010020

You can use a Series for comparison.

Either ensure that the value matches the Series (df.eq(s)), or (|) that the Series contains a empty string (s.eq('')). Broadcasting magic will do the rest ;)

s = pd.Series(l, index=df.columns)

df2 = df[(df.eq(s)|s.eq('')).all(1)]

Example with ['A', 'B', '', '', 123]:

# input
   a  b  c  d    e
0  A  B  C  D  123
1  A  X  C  D  456

# output
   a  b  c  d    e
0  A  B  C  D  123
user_163434640
Related