Pandas.cut VS df.describe()

user_19100837 0
Viewed 38

I would like to group the datas with 4 ranges,and I used Pandas.cut to bin,here is my code and result

cut image

Then I used df.describe() and I found the ranges with the edges are different with pd.cut,why?

describe

pd.cut is [(2.719, 3.042] < (3.042, 3.365] < (3.365, 3.688] < (3.688, 4.01]]

df.describe() is

min         2.720000
25%         3.110000
50%         3.210000
75%         3.320000
max         4.010000
pandas dataframe python
1 Answers

Your cut divides the range into 4 equal-width bins, whereas describe uses quartiles. Only for uniformly distributed data both would result in the same subdivisions.

Example:

import pandas as pd
import numpy as np

df = pd.DataFrame({'uniform': np.random.rand(1_000_000), 'normal': np.random.randn(1_000_000)})

with np.printoptions(formatter={'float': '{:.3f}'.format}):
    print( 'uniform:\n'
           f'   {df.uniform.describe().iloc[3:].values}\n'
           f'   {pd.cut(df.uniform, 4).dtype.categories.to_tuples().to_list()}')
    print( 'normal:\n'
           f'   {df.normal.describe().iloc[3:].values}\n'
           f'   {pd.cut(df.normal, 4).dtype.categories.to_tuples().to_list()}')

Output:

uniform:
   [0.000 0.250 0.499 0.750 1.000]
   [(-0.001, 0.25), (0.25, 0.5), (0.5, 0.75), (0.75, 1.0)]
normal:
   [-4.908 -0.675 0.001 0.674 5.082]
   [(-4.918, -2.411), (-2.411, 0.0867), (0.0867, 2.584), (2.584, 5.082)]
user_39443220
Related