Pytorch: Weighted Covariance

Question

I am trying to implement a PyTorch covariance matrix operator. However, I notice the results are not the same between the Numpy implementation and my attempt, yet I do not understand why.

I define the Bessel-corrected weighted covariance matrix as:

I define the weighted mean as:

I compare the NumPy method and my method as follows:

import numpy as np
import torch

torch.set_printoptions(precision=8)

x = np.random.randn(1000, 3)*1000
w = np.abs(np.random.randn(1000))*1000

x_torch = torch.DoubleTensor(x)
w_torch = torch.DoubleTensor(w)

#calculate weighted means
m_w = torch.sum(x_torch.T*w_torch, axis=1)/torch.sum(w_torch)
m_w_np = np.average(x, axis=0, weights=w)

#calculate weighted covariance matrix
Q = (x_torch-m_w).T
cov_w = (1.0 / (torch.sum(w_torch) - 1))*(w_torch*Q).mm(Q.T)
cov_w_np = np.cov(x.T, aweights=w.T)

print("WEIGHTED MEAN")
print("NUMPY = {0}

TORCH = {1}

DIFFERENCE={2}".format(m_w_np, m_w.numpy(), m_w_np-m_w.numpy())) 
print("")
print("")
print("WEIGHTED COVARIANCE")
print("NUMPY = {0}

TORCH = {1}

DIFFERENCE={2}".format(cov_w_np, cov_w.numpy(),cov_w_np-cov_w.numpy()))

This yields the following output:

WEIGHTED MEAN
NUMPY = [-21.10537208  -7.70801723  64.4034329 ]

TORCH = [-21.10537208  -7.70801723  64.4034329 ]

DIFFERENCE=[-7.10542736e-15 -1.77635684e-15  1.42108547e-14]


WEIGHTED COVARIANCE
NUMPY = [[ 989468.17457696   13620.54885133   10723.87790683]
         [  13620.54885133  953966.92486133   21407.69378841]
         [  10723.87790683   21407.69378841 1019646.81044077]]

TORCH = [[ 987952.51042915   13599.68493868   10707.45110536]
         [ 13599.68493868  952505.64141296   21374.90155234]
         [ 10707.45110536   21374.90155234 1018084.91875621]]

DIFFERENCE=[[1515.6641478    20.86391265   16.42680147]
            [  20.86391265 1461.28344838   32.79223607]
            [  16.42680147   32.79223607 1561.89168456]]

Pytorch: Weighted Covariance

0 Answers