Calculate percentiles if we have probability density function data as x and y

user_893003 0
Viewed 1541

I have data extracted from a pdf graph where x represents incubation times and y is the density in a csv file. I would like to calculate the percentiles, such as 95%. I'm a bit confused, should I calculate the percentile using the x values only, i.e., using np.precentile(x, 0.95)?

data in plot: enter image description here

kernel-density probability distribution percentile python
2 Answers

Here is some code which uses np.trapz (as proposed by @pjs). We take x and y arrays, assume it is PDF so first we normalize it to 1, an then start searching backward till we hit 0.95 point. I've made up some multi-peak function

import numpy as np
import matplotlib.pyplot as plt

N = 1000

x = np.linspace(0.0, 6.0*np.pi, N)
y = np.sin(x/2.0)/x # construct some multi-peak function
y[0] = y[1]
y = np.abs(y)

plt.plot(x, y, 'r.')
plt.show()

# normalization
norm = np.trapz(y, x)
print(norm)

y = y/norm
print(np.trapz(y, x)) # after normalization

# now compute integral cutting right limit down by one
# with each iteration, stop as soon as we hit 0.95
for k in range(0, N):
    if k == 0:
        xx = x
        yy = y
    else:
        xx = x[0:-k]
        yy = y[0:-k]
    v = np.trapz(yy, xx)
    print(f"Integral {k} from {xx[0]} to {xx[-1]} is equal to {v}")
    if v <= 0.95:
        break
user_40446960

I have test both @Severin Pappadeux method and np.percentile and bith gave me the same result for 95 percentile

Here code of @Severin Pappadeux but with the data I used:

import numpy as np
import matplotlib.pyplot as plt


x = [ 5.  ,  5.55,  6.1 ,  6.65,  7.2 ,  7.75,  8.3 ,  8.85,  9.4 ,
      9.95, 10.5 , 11.05, 11.6 , 12.15, 12.7 , 13.25, 13.8 , 14.35,
      14.9 , 15.45, 16.  ]
y = [0.03234577, 0.03401444, 0.03559847, 0.03719304, 0.03890566,
     0.04084201, 0.04309067, 0.04570878, 0.04871024, 0.05205822,
     0.05566298, 0.05938525, 0.06304516, 0.06643575, 0.06933978,
     0.07154828, 0.07287886, 0.07319211, 0.0724044 , 0.0704957 ,
     0.0675117 ] 

N = len(x)

y[0] = y[1]
y = np.abs(y)

plt.plot(x, y, 'r.')
plt.show()

# normalization
norm = np.trapz(y, x)
print(norm)

y = y/norm
print(np.trapz(y, x)) # after normalization

# now compute integral cutting right limit down by one
# with each iteration, stop as soon as we hit 0.95
for k in range(0, N):
    if k == 0:
       xx = x
       yy = y
    else:
       xx = x[0:-k]
       yy = y[0:-k]
     v = np.trapz(yy, xx)
print(f"Integral {k} from {xx[0]} to {xx[-1]} is equal to {v}")

    if v <= 0.95:
       break

# Outputs = 
# 0.6057000785
# 1.0 
# Integral 0 from 5.0 to 16.0 is equal to 1.0
# Integral 1 from 5.0 to 15.45 is equal to 0.9373418687777172

and when I used np.percentile() on x as @Zeek suggests:

np.percentile(x, 95)
# Output= 15.45

So, both methods gave me 15.45 as the 95 percentile of x

user_8930030
Related