Can't select Pandas DataFrame by quantiles

user_10060701 0
Viewed 49

I used Pandas qcut function to sort a column of values into quantiles but I can't filter the DataFrame by those quantiles, here goes an example:

df = pd.DataFrame(data = [i for i in range(100)], columns = ['values'])
df['bins'] = pd.qcut(df['values'], q = 10)

This gives me this DataFrame:

Dataframe

But when I try to filter by some decile:

df[df['bins'] == (-0.001, 9.9]]

I get: SyntaxError: invalid syntax

Changing the interval to a string, like df[df['bins'] == '(-0.001, 9.9]'] just returns me an empty DataFrame, so it doesn't help either. How should I go about this?

pandas dataframe python
1 Answers

The pandas.qcut method return a Categorical series with an pandas.IntervalIndex. To index into that series you need to query using pandas.Intervals:

df = pd.DataFrame(data = [i for i in range(100)], columns = ['values'])
df['bins'] = pd.qcut(df['values'], q = 10)
df.loc[df['bins'] == pd.Interval(-0.001, 9.9)]

The reason you get a syntax error using df['bins'] == (-0.001, 9.9] is that Python expects brackets of the same type to match up. When printing the dataframe it does show exactly that sequence, because that matches the conventional notation, but it is just the string representation of the pd.Interval object that is actually in the dataframe.

user_43985950
Related