Why am I getting a divide by zero error here?

Question

So I'm following along this tutorial in the docs on custom datasets. I'm using the MNIST dataset instead of the fancy one in the tutorial. This is the extension of the Dataset class I wrote:

class KaggleMNIST(Dataset):

    def __init__(self, csv_file, transform=None):
        self.pixel_frame = pd.read_csv(csv_file)
        self.transform = transform

    def __len__(self):
        return len(self.pixel_frame)

    def __getitem__(self, index):
        if torch.is_tensor(index):
            index = index.tolist()

        image = self.pixel_frame.iloc[index, 1:]
        image = np.array([image])

        if self.transform:
            image = self.transform(image)

        return image

It works, until I try to use a transform on it:

tsf = transforms.Compose([transforms.ToTensor(), 
                          transforms.Normalize((0.5,), (0.5,))
                         ])
                          
trainset = KaggleMNIST('train/train.csv', transform=tsf)

image0 = trainset[0]

I've looked at the stack trace, and it seems like the normalization is happening in this line of code:

c:\program files\python38\lib\site-packages	orchvision	ransforms\functional.py in normalize(tensor, mean, std, inplace)
--> 218     tensor.sub_(mean[:, None, None]).div_(std[:, None, None])

So I don't get why there is divide by zero since std should be 0.5, nowhere remotely close to a small value.

Thanks for your help!

EDIT:

This does not answer my question, but I discovered that if I change these lines of code:

image = self.pixel_frame.iloc[index, 1:] 
image = np.array([image])

to

image = self.pixel_frame.iloc[index, 1:].to_numpy(dtype='float64').reshape(1, -1)

Essentially, making sure the datatype was float64 fixed the problem. I'm still not sure why the problem existed in the first place, so I'd still be happy for a well-explained answer!

user_6210807 · Answer

The dtype of the data read is int64

img = np.array([pixel_frame.iloc[0, 1:]])
img.dtype
# output
dtype('int64')

This forces the mean and std to be converted to int64 and as std is 0.5, it becomes 0, and raises the following error:

>>> tsf(img)
ValueError: std evaluated to zero after conversion to torch.int64, leading to division by zero.

It's because the mean and std are converted to dtype of the dataset during normalization.

def normalize(tensor, mean, std, inplace=False):
    ...
    dtype = tensor.dtype
    mean = torch.as_tensor(mean, dtype=dtype, device=tensor.device)
    std = torch.as_tensor(std, dtype=dtype, device=tensor.device)
    if (std == 0).any():
        raise ValueError('std evaluated to zero after conversion to {}, leading to division by zero.'.format(dtype))

That's why converting the dtype to float fixes the error.

Why am I getting a divide by zero error here?

1 Answers