Pandas filter on categorical intervals

user_1892308 0
Viewed 2304

I create a dataframe and categorize one column as intervals :

df_test = pd.DataFrame({'col': [0,1,2,3,4,5,6]})
df_test['cat']= pd.cut(df_test['col'],[-1.,0.,3.,10.])
df_test

        col     cat
    0   0   (-1.0, 0.0]
    1   1   (0.0, 3.0]
    2   2   (0.0, 3.0]
    3   3   (0.0, 3.0]
    4   4   (3.0, 10.0]
    5   5   (3.0, 10.0]
    6   6   (3.0, 10.0]

Now I want to filter this dataframe using the cat column :

df_test[df_test['cat'] == pd.Interval(left=1., right=2.)]

    col     cat
1   1   (0.0, 3.0]
2   2   (0.0, 3.0]
3   3   (0.0, 3.0]

How come that checking equality with (1., 2.] yields this result ? I was expecting to get an empty result as that interval doesn't exist in the dataframe.

Am I supposed to filter using a different method ?

pandas python
3 Answers

For exact matching is possible use hack solution - convert both to strings:

a = df_test[df_test['cat'].astype(str) == str(pd.Interval(left=1., right=2.))]

Or use apply:

a = df_test[df_test['cat'].apply(lambda x: x == pd.Interval(left=1., right=2.))]
print (a)
Empty DataFrame
Columns: [col, cat]
Index: []

More information why this is implemented for check membership is here

user_29010020

Your function is working, the syntax means right now that he filters everything, which intervall is in (1, 2), so in your case 0.0 until 3.0 has 1-2 inside, so he gives back a true, if you try: df_test[df_test['cat'] == pd.Interval(left=10, right=20)] you will get an empty datframe

If you want to see an excact matching, maybe it is better to split the intervall?

user_102329320

list comprehension provides the result you expect:

[i == pd.Interval(1,2) for i in df['cat']]

Output:

[False, False, False, False, False, False, False]
user_70937410
Related