Pytorch code error.... ' NoneType' object has no attribute 'zero_' 'unsupported operand type(s) for *: 'float' and 'NoneType''

user_13797535 0
Viewed 1138
lr=0.001

x=np.linspace(-6,6,120)
y=0.5+3*x-x**2+np.exp(-0.4*x)
x=torch.from_numpy(x)
y=torch.from_numpy(y)


w0=torch.tensor(0.1,requires_grad=True)
w1=torch.tensor(0.1,requires_grad=True)
w2=torch.tensor(0.1,requires_grad=True)
w3=torch.tensor(0.1,requires_grad=True)

Y=w0+w1*x+w2*x**2+torch.exp(w3*x)
#opt=torch.optim.Adam([w0,w1,w2,w3],0.001)

L=nn.MSELoss()

for i in range(20):
  opt.zero_grad()
  err=L(Y,y)
  print(err)
  err.backward(retain_graph=True)
  with torch.no_grad():
    w0=w0-lr*w0.grad
    w1=w1-lr*w1.grad
    w2=w2-lr*w2.grad
    w3=w3-lr*w3.grad  
    
    w0.grad.zero_()
    w1.grad.zero_()
    w2.grad.zero_()
    w3.grad.zero_()

from the line w0 = w0 - lr * w0.grad:

===> unsupported operand type(s) for *: 'float' and 'NoneType'

from the line w0.grad.zero_():

===> 'NoneType' object has no attribute 'zero_'

comes up.

How should I fix it?

And

x=np.linspace(-6,6,120)
y=0.5+3*x-x**2+np.exp(-0.4*x)
x=torch.from_numpy(x)
y=torch.from_numpy(y)


w0=torch.tensor(0.1,requires_grad=True)
w1=torch.tensor(0.1,requires_grad=True)
w2=torch.tensor(0.1,requires_grad=True)
w3=torch.tensor(0.1,requires_grad=True)

Y=w0+w1*x+w2*x**2+torch.exp(w3*x)
opt=torch.optim.Adam([w0,w1,w2,w3],0.001)

L=nn.MSELoss()

for i in range(20):
  opt.zero_grad()
  err=L(Y,y)
  print(err)
  err.backward(retain_graph=True)
  opt.step()

If I try this code, then err is not updated. What is the problem? and how should I fix it

  1. Further, should the input of nn.MSELoss() be torch.double? Sometimes I get expected dtype Double but got dtype Float error.

What should be the type of parameters w0, w1, ...?

pytorch
1 Answers
  1. You should do the following.
for i in range(20):
    Y=w0+w1*x+w2*x**2+torch.exp(w3*x)
    err=L(Y,y)
    print(err)
  
    err.backward(retain_graph=True)
    with torch.no_grad():
        w0.add_(-lr*w0.grad)
        w1.add_(-lr*w1.grad)
        w2.add_(-lr*w2.grad)
        w3.add_(-lr*w3.grad) 
    
        w0.grad.zero_()
        w1.grad.zero_()
        w2.grad.zero_()
        w3.grad.zero_()

Please note, you are performing Y=w0+w1*x+w2*x**2+torch.exp(w3*x) outside of the for loop which is wrong, and as a result err was not updating.

  1. dtype=torch.float64 should be fine.
user_53523990
Related