How to compare the sizes of confidence intervals in python

user_10771559 0
Viewed 155

I have a dataframe and I am finding the confidence intervals across each row. My actual dataframe is hundreds of rows long, but here is an example:

df = pd.DataFrame({'nums_1': [1, 2, 3], 'nums_2': [1, 1, 5], 'nums_3' : [8,7,9]})    


df['CI']=df.apply(lambda row: stats.t.interval(0.95, len(df)-1, 
loc=np.mean(row), scale=stats.sem(row)), axis=1).apply(lambda x: np.round(x,2))

I also want to calculate the width of each confidence interval. I tried the the following, but it did not work

df['width']=df.apply(lambda row: stats.t.interval(0.95, len(df)-1, 
loc=np.mean(row), scale=stats.sem(row)), axis=1)[1] - df.apply(lambda row: 
stats.t.interval(0.95, len(df)-1, 
loc=np.mean(row), scale=stats.sem(row)), axis=1)[0]
pandas dataframe numpy python
1 Answers

IIUC, you want to compute the difference between upper from lower in the confidence interval, you can try this:

df['CI'].apply(lambda x: x[1] - x[0])

If you have this:

>>> from scipy import stats
>>> import numpy as np
>>> df = pd.DataFrame({'nums_1': [1, 2, 3], 'nums_2': [1, 1, 5], 'nums_3' : [8,7,9]})
>>> df['CI']=df.apply(lambda row: stats.t.interval(0.95, len(df)-1, loc=np.mean(row), scale=stats.sem(row)), axis=1).apply(lambda x: np.round(x,2))
>>> df['CI']
0    [-6.71, 13.37]
1    [-4.65, 11.32]
2    [-1.92, 13.26]
Name: CI, dtype: object

you get this:

>>> df['width'] = df['CI'].apply(lambda x: x[1] - x[0])
>>> df
    nums_1  nums_2  nums_3  CI              width
0   1       1       8       [-6.71, 13.37]  20.08
1   2       1       7       [-4.65, 11.32]  15.97
2   3       5       9       [-1.92, 13.26]  15.18
user_17405770
Related