PyTorch's Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same but my data has been pushed to GPU

Viewed 3739

I receive the error:

RuntimeError: Input type (torch.FloatTensor) and weight type (torch.cuda.FloatTensor) should be the same

yet I made sure to send my data as well as my model on the GPU. Anyone can help ?

My code is:

net.cuda()

'''
print('pyTorch style  summay: ',net)
print('Keras style summary:\n')
summary(net,(2,128,128))
'''

criterion=nn.MSELoss()
#optimizer = optim.SGD(net.parameters(), lr=learning_rate, momentum=0.9)
learning_rate = 1e-4
optimizer = torch.optim.Adam(net.parameters(), lr=learning_rate)
print('\nLossFun=',str(criterion))

hf=h5py.File(fn,'r')
print(hf['trainingset'])
tr=np.array(hf['trainingset'])
trtg=np.array(hf['targetsTraining'])
hf.close()


tr = np.moveaxis(tr,3,2)
trtg = np.moveaxis(trtg,3,2)

tr = torch.FloatTensor(tr)
trtg = torch.FloatTensor(trtg)

tr.cuda()
trtg.cuda()


batch_size=16
epochs=2
# run the main training loop
for epoch in range(epochs):
    for batch_idx in range(batch_size):#batch_idx, (data, target) in enumerate(train_loader):
        data =  tr[batch_idx:batch_idx+batch_size-1,:,:,:] 
        target =  trtg[batch_idx:batch_idx+batch_size-1,:,:,:] 
        data, target = Variable(data), Variable(target)
        # resize data from (batch_size, 1, 28, 28) to (batch_size, 28*28)
        #data = data.view(-1, 28*28)
        optimizer.zero_grad()
        net_out = net(data)
        loss = criterion(net_out, target)
        loss.backward()
        optimizer.step()
        batch_idx += 1
        if batch_idx % log_interval == 0:
            print('Train Epoch: {} [{}/{} ({:.0f}%)]\tLoss: {:.6f}'.format(epoch, batch_idx * len(data), len(train_loader.dataset),100. * batch_idx / len(train_loader), loss.data[0]))

I don't understand why tr.cuda() and trtg.cuda() is not enough! How can I force them to go to cuda?

1 Answers

Tensor.cuda() doesn't actually change the tensor in place. It creates a copy of the tensor on the GPU and returns the copy. With this in mind you probably want

tr = tr.cuda()
trtg = trtg.cuda()

This is actually different than nn.Module.cuda() which is performed in-place and actually modifies the registered parameters and buffers of the module.

Related