Pandas pd.Series().var() function is giving different output when compared with calculating std manually. Any Idea what's happening wrong here?

user_9358685 0
Viewed 180

I am trying to calculate the standard deviation manually instead of using the pandas.Series().std() method but getting difference in output. Please help.

Manual Variance Calculation

values = pd.read_csv('Values.csv')

mom_1 = pd.Series(values['col_name'].mean() - values['col_name'])

mom_2 = mom_1*mom_1

print(np.sqrt(np.sum(mom_2.values,dtype = np.float64)/ values.shape[0] - 1))

Output 1 - 10773.042044307498

Pandas Variance Calculation

print(np.sqrt(pd.Series(values['col_name'].astype(np.float64)).var()))

Output 2 - 10773.042044307516

Tried replicating pandas nanvar function from pandas.core.nanops pandas nanvar implementaton line 711 but still getting output 1 only. find the Values.csv here

pandas==1.0.1 numpy==1.17.0

Any Idea what's going wrong here?

pandas variance numpy python
1 Answers

it's because of a library called bottleneck, which pandas use for the faster calculation. Since pandas is using it with the default settings, turning it off now results in the same output.

user_93586850
Related