Numpy not producing desired sample variance value

user_10368128 0
Viewed 604

I have a list for which I like to calculate sample variance. When I use numpy.var I get a different result from the function I defined.

Can someone please help me understand what I am missing? 

my_ls = [227, 222, 218, 217, 225, 218, 216, 229, 228, 221]


def calc_mean(ls):

        sum_tmp = 0
        for i in ls:
                sum_tmp = sum_tmp + i

        return round(sum_tmp/len(ls), 2)

def calc_var(ls):

        tmp_mean = calc_mean(ls)

        tmp_sum = 0
        for i in ls:
                tmp_sum = tmp_sum + ((i - tmp_mean)**2)

        return round(tmp_sum/(len(ls)-1), 2)


calc_var(my_ls)
>>> 23.66

np.var(my_ls)
>>> 21.29

23.66 is my desired value.

variance numpy python
4 Answers

The denominator in the case of np.var(my_ls) by default is the total sample size (N).

You can use the Delta Degrees of Freedom (ddof) parameter in numpy to show you're calculating the sample variance by setting ddof = 1 for the mean degree of freedom. I.e. your denominator now becomes (N-1)

np.var(my_ls,ddof=1)
>>> 23.66
user_117433180

You can use the ddof parameter of np.var(), which stands for "degrees of freedom":

np.var(my_ls, ddof=1)
# 23.65555555555555

to get you to the desired result.

Essentially, you divide your sum of squares by n - 1, while np.var() divides by n - ddof, which defaults to 0. A discussion on these topics can be found on Wikipedia.

user_52183540

You are using the unbiased formula for the variance, i.e. you are dividing the summatory by N-1, while np.var seems to compute the variance normalizing by the total number of elements in the vector.

Check for example here, section "Sample variance".

user_131562610

your function calc_var(ls) doesn't implement variance formula:

The variance is the average of the squared deviations from the mean, i.e., var = mean(abs(x - x.mean())**2).

you can use:

def calc_var(ls):

        tmp_mean = calc_mean(ls)

        means = []
        for i in ls:
                means.append((i - tmp_mean)**2)

        var = calc_mean(means)
        return round(var, 2)

print(calc_var(my_ls))
print(np.var(my_ls))

output:

21.29
21.29
user_31615750
Related