How to handle 'nan' values in PyTorch while training a neural network with a new parameter in exponent?

Viewed 191

I am running a program that learns parameter tensor x. But, i changed it to x^y by adding a new parameter y to my model.

The later results in nan values. You can check the below for example

>>>import torch
>>>x = torch.randn(4,3)
>>>y = torch.randn(1)
>>print(x)
tensor([[-0.7662,  0.7113, -0.8803],
        [-1.0947,  0.4314,  2.2009],
        [-1.4202,  0.5253, -0.5965],
        [-0.1162,  1.3597,  1.1211]])
>>print(y)
tensor([0.6968])
>>print(x ** y)
tensor([[   nan, 0.7887,    nan],
        [   nan, 0.5567, 1.7327],
        [   nan, 0.6385,    nan],
        [   nan, 1.2387, 1.0829]])

If I try to initialize y to integers, then it is giving the following error

RuntimeError: Only Tensors of floating point and complex dtype can require gradients

How to handle this issue in-order to train my network?

0 Answers
Related