Filter on list of list pandas

user_7936119 0
Viewed 701

I have a dataframe that that contains a list for one of the column. Example

df.loc[x,column] 
#returns [1,4,5,6]

I would like to select the row when the value is either 1 or 2 or 3. I was thinking doing this in a list so I can filter like this : 

filter = [1,2,3]
df_new = df[df[column] is in filter]

However, it returns me "The truth value of a Series is ambiguous."

if I try this : 

filter = [1,2,3]
df_new = df[df[column].isin(filter)]

It returns me <built-in method view of numpy.ndarray object at 0x000001E0CDF09440> returned a result with an error set

I try to use any but it doesn't seem to work here. I also had a look on the different thread posted with np.select or using the pipe "|" but it doesn't really apply here. Does it ? Any tips ?

simplify example dataframe: 

   column
0  [1,2]
1  [3,4]
2  [5]
3  []
4  [7,8]
5  [5,3]

Expected output (filtering for 1, 2 or 3): 

   column
0  [1,2]
1  [3,4]
5  [5,3]
pandas python-3.x python
2 Answers

You can use Series.apply to apply a custom function on the column which creates a boolean mask. This custom function checks each individual item of series to find if there are any elements in this item that are also present in the filter list.

Use:

lst = [1, 2, 3] # filter list
m = df[column].apply(lambda s: len(set(s) & set(lst)) > 0)
print(df[m])

This prints:

       column
0      [1, 2]
1      [3, 4]
5      [5, 3]
user_128331660

Unless you need something more complex, you're pretty close to how you do it in pandas

df.loc[df[column].isin([filter])]
user_125075250
Related