I am working on a task to obtain distribution parameters from a Pareto distribution and then generate values cdf values from the Pareto distribution.
For example
pareto_fit = stats.pareto.fit(pareto_df['Wait_Times'], 1, loc = 0, scale = 1, floc=0)
returns a three-parameter list with the following precision (16 decimal places):
(0.9174317832009571, 0, 4.45113004935033)
However, when I calculate the CDF values based on my dataset
frozen_pareto = stats.pareto(b = pareto_shape, scale = pareto_scale)
pareto_Fx_i = frozen_pareto.cdf(pareto_df['Wait_Times'])
I get the following values:
[0. 0.00602124 0.01052278 0.01120112 0.01828878 0.02379275
0.02559216 0.0357929 0.04475035 0.04578789 0.04904739 0.05744869
0.07884679 0.08155561 0.10014234 0.10582278 0.10729736 0.11032082
0.11942508 0.15042217 0.15051626 0.15583477 0.18819108 0.28507186
0.38839434 0.40743893 0.41805445 0.43426831 0.44341763 0.46636651
0.49741257 0.5204878 0.53165406 0.58486948 0.5917222 0.65856103
0.668838 0.67332618 0.68927657 0.73241305 0.77687016 0.88356968
0.91663373 0.92539471 0.92933826 0.93299396 0.93632732 0.94689438
0.94909401 0.97091551 0.98906386 0.99751471]
The main concern is the eight decimal places of precision returned (from what looks like a NumPy array. The first value '0' is problematic for downstream calculations (i.e. I need to take the log of these cdf values). Is there a way to increase the decimal places returned from the cdf calculations to say 16 like in the parameter list above?
I note that when I compute the cdf values in R I get the following values using the EnvStats package:
library(EnvStats)
> ppareto(pareto.df$Wait_Times, 4.45113004935, shape = 0.917431783201)
6.8167693711985e-14 6.0212407771670e-03 1.0522776389728e-02 1.1201124356793e-02 1.8288781136547e-02 2.3792752873401e-02
2.5592157258839e-02 3.5792904811076e-02 4.4750348533511e-02 4.5787894409251e-02 4.9047386512592e-02 5.7448687480950e-02
7.8846790983340e-02 8.1555606794742e-02 1.0014233586481e-01 1.0582277809221e-01 1.0729736014552e-01 1.1032082062863e-01
1.1942508142076e-01 1.5042216535740e-01 1.5051625805110e-01 1.5583477140579e-01 1.8819108156064e-01 2.8507186327579e-01
3.8839433881133e-01 4.0743892955800e-01 4.1805444580571e-01 4.3426831018132e-01 4.4341763117122e-01 4.6636650951443e-01
4.9741257267683e-01 5.2048780232284e-01 5.3165406286277e-01 5.8486948075320e-01 5.9172219812372e-01 6.5856102783587e-01
6.6883800320072e-01 6.7332617524269e-01 6.8927657280333e-01 7.3241304741455e-01 7.7687015799430e-01 8.8356968182618e-01
9.1663373303915e-01 9.2539471393846e-01 9.2933825721611e-01 9.3299395668682e-01 9.3632732124269e-01 9.4689438060868e-01
9.4909401359761e-01 9.7091551200033e-01 9.8906386116803e-01 9.9751470667794e-01
my_data = [4.4511300493503, 4.4805284392512, 4.5027512569270, 4.5061184044368, 4.5415906356577, 4.5695082533874,
4.5787068114938, 4.6315314794649, 4.6788902848223, 4.6844359380410, 4.7019401067623, 4.7476403532855,
4.8679770339429, 4.8836286408396, 4.9936809034313, 5.0282692216452, 5.0373231932895, 5.0559854107284,
5.1129902581507, 5.3166586135152, 5.3173005144204, 5.3538265968269, 5.5868313056793, 6.4168838229709,
7.6070487961840, 7.8739205299675, 8.0306066058598, 8.2817984566921, 8.4302996165132, 8.8262279014230,
9.4221368493479, 9.9174137219138, 10.1754186585690, 11.6050958855081, 11.8175710845523, 14.3601404431787,
14.8465590392403, 15.0690305968270, 15.9141068218137, 18.7297984492720, 22.8318960824204, 46.3935489859590,
66.7713319465588, 75.3616899071995, 79.9573773033068, 84.7237613685393, 89.5695808475361, 109.1606887601360,
114.3118679797660, 210.4157548318590, 611.0927497881170, 3072.6055750405599]