pd.qcut is returning negative values

user_4080815 0
Viewed 857

Here is a simple sample serie of data :

sample
Out[2]: 
0    0.047515
1    0.026392
2    0.024652
3    0.022854
4    0.020397
5    0.000087
6    0.000087
7    0.000078
8    0.000078
9    0.000078

The lower value is 0.000078 and max value is 0.047515. When I use the qcut function on it, the results give me negative data on my categories.

pd.qcut(sample, 4)
Out[31]: 
0         (0.0242, 0.0475]
1         (0.0242, 0.0475]
2         (0.0242, 0.0475]
3         (0.0102, 0.0242]
4         (0.0102, 0.0242]
5       (8.02e-05, 0.0102]
6       (8.02e-05, 0.0102]
7    (-0.000922, 8.02e-05]
8    (-0.000922, 8.02e-05]
9    (-0.000922, 8.02e-05]
Name: data, dtype: category
Categories (4, interval[float64]): [(-0.000922, 8.02e-05] < (8.02e-05, 0.0102] < (0.0102, 0.0242] < (0.0242, 0.0475]]

Is it an expected behavior ? I thought that I would find my min and max as lower and upper bound of my categories.

(I use pandas 0.22.0 and python-2.7)

pandas python-2.7 python
1 Answers

This happens because the binning procedure subtracts .001 from the lowest value in your range. If the edges of a bin == an exact number in your series, it is unclear which bin the number should be placed into. Thus, it makes sense to slightly adjust the min and max before creating the qtiles.

See lines 210-213 in the source code for pd.cut. https://github.com/pandas-dev/pandas/blob/v0.23.4/pandas/core/reshape/tile.py#L210-L213 

0.000078 -.001
Out[21]: -0.0009220000000000001
user_77479750
Related