Using scipy for T-testing returning NaN error code, even with no NaN values present

Viewed 200

I was working on a code that would calculate the t-values and p-values of data from an excel spreadsheet based on heights from different genders (1 and 0 are used to differentiate the genders in the excel sheet). I also need to use different ranges of heights, so the first 10, then first 20 and lastly the first 30 heights. However, it always returns "nan" instead of a number, even when I write "nan_policy='omit'".I am aware of a user on here who has had the same issue as me, however, he was using pandas, which I am not. I am using spyder4 and the latest version of Anaconda. I'm also using python version 3.8.3 and version 1.5.0 of scipy. Here is the code:

import numpy as np
import scipy.stats

array = np.loadtxt(r'C:\filepath\Body-Data.csv', skiprows = 1, delimiter=',' )

slice10 = slice(0,10)
slice20 = slice(0,20)
slice30 = slice(0,30)

men_height = []
women_height = []

for i in range(8239):
    if array[i,0] == 0:
        women_height.append(array[i,2])
    elif array[i,0] == 1:
        men_height.append(array[i,2])
        
w_height10 = women_height[slice10]
w_height20 = women_height[slice20]
w_height30 = women_height[slice30]

m_height10 = men_height[slice10]
m_height20 = men_height[slice20]
m_height30 = men_height[slice30]

w_mean10 = np.mean(w_height10) 
w_mean20 = np.mean(w_height20) 
w_mean30 = np.mean(w_height30) 
    
m_mean10 = np.mean(m_height10)
m_mean20 = np.mean(m_height20)
m_mean30 = np.mean(m_height30)

t_statistic1, p_value1 = scipy.stats.ttest_ind(m_mean10, w_mean10, nan_policy='omit')
print("this is the t-statistic for the first 10 heights of women and men: \n", t_statistic1)
print("this is the p-value for the first 10 heights of women and men: \n", p_value1)


t_statistic2, p_value2 = scipy.stats.ttest_ind(m_mean20, w_mean20, nan_policy='omit')
print("this is the t-statistic for the first 20 heights of women and men: \n", t_statistic2)
print("this is the p-value for the first 20 heights of women and men: \n", p_value2)


t_statistic3, p_value3 = scipy.stats.ttest_ind(m_mean30, w_mean30, nan_policy='omit')
print("this is the t-statistic for the first 30 heights of women and men: \n", t_statistic3)
print("this is the p-value for the first 30 heights of women and men: \n", p_value3)

My output is:

this is the t-statistic for the first 10 heights of women and men: 
 nan

this is the p-value for the first 10 heights of women and men: 
 nan

this is the t-statistic for the first 20 heights of women and men: 
 nan

this is the p-value for the first 20 heights of women and men: 
 nan

this is the t-statistic for the first 30 heights of women and men: 
 nan

this is the p-value for the first 30 heights of women and men: 
 nan
1 Answers

The first two arguments to scipy.stats.ttest_ind must be the data sets that are to be compared, not the means of the data sets. Change this line

t_statistic1, p_value1 = scipy.stats.ttest_ind(m_mean10, w_mean10, nan_policy='omit')

to

t_statistic1, p_value1 = scipy.stats.ttest_ind(m_height10, w_height10, nan_policy='omit')

(If there are no nans in the input, you can remove the argument nan_policy='omit'.)

For other variations of computing the t-statistic, see Perform 2 sample t-test

Related