Pytorch - RuntimeError: Expected object of scalar type Long but got scalar type Float for argument #2 'target' in call to _thnn_nll_loss_forward

user_9969041 0
Viewed 284

I was trying & experimenting something with PyTorch, where I created my own inputs & targets. I fed these inputs to the model (which is a basic ANN with 2 hidden layers, nothing wrong with that). But for some reason I am not being able to calculate the CrossEntropyLoss(). I am not being able to figure out why. I know some of the other questions on StakcOverflow have the same title of mine or have a similar problem. I have gone through that but nothing worked out for me. Alot of people had an issue with the dataset, which does not seem to be the problem with me.

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

class Net(nn.Module):
    def __init__(self) -> None:
        super(Net, self).__init__()
        self.layer1 = nn.Linear(2, 10)
        self.layer2 = nn.Linear(10, 1)

    def forward(self, x):
        x = F.relu(self.layer1(x))
        x = self.layer2(x)
        return x
    
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = Net().to(device=device)
loss_fn = nn.CrossEntropyLoss()
learning_rate = 1e-3
epochs = 20
optimizer = optim.Adam(model.parameters(), lr=learning_rate)
inputs = torch.Tensor([
    [0,0],
    [0,1],
    [1,0],
    [1,1]
], ).to(device=device)

targets = torch.Tensor([
    0,
    1,
    1,
    0
]).to(device=device)

model.train()
for epoch in range(epochs):

    pred_output = model(inputs)
    print(pred_output.dtype)
    print(targets.dtype)
    loss = loss_fn(pred_output, targets)
    
    optimizer.zero_grad()
    loss.backward()
    optimizer.step()
    print()
    break

The error that I see is,

torch.float32
torch.float32
Traceback (most recent call last):
  File ".\main.py", line 57, in <module>
    loss = loss_fn(pred_output, targets)
  File "C:\Users\user\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 889, in _call_impl
    result = self.forward(*input, **kwargs)
  File "C:\Users\user\anaconda3\lib\site-packages\torch\nn\modules\loss.py", line 1047, in forward
    return F.cross_entropy(input, target, weight=self.weight,
  File "C:\Users\user\anaconda3\lib\site-packages\torch\nn\functional.py", line 2693, in cross_entropy
    return nll_loss(log_softmax(input, 1), target, weight, None, ignore_index, None, reduction)
  File "C:\Users\user\anaconda3\lib\site-packages\torch\nn\functional.py", line 2388, in nll_loss
    ret = torch._C._nn.nll_loss(input, target, weight, _Reduction.get_enum(reduction), ignore_index)
RuntimeError: Expected object of scalar type Long but got scalar type Float for argument #2 'target' in call to _thnn_nll_loss_forward
loss cross-entropy pytorch python
1 Answers

I could replicate you error using this code.

import torch.nn as nn
loss = nn.CrossEntropyLoss()
input = torch.randn(3, 5, requires_grad=True)
target = torch.tensor([1., 2., 3.])
loss(input, target)

Error:

RuntimeError: expected scalar type Long but found Float

changed the datatype of target to target = torch.tensor([1., 2., 3.], dtype=torch.long) and everything worked fine. I believe the target variable does require long datatype because changing the input to float will also work.

#this will also work
input = torch.randn(3, 5, requires_grad=True, dtype=torch.float)
target = torch.tensor([1., 2., 3.], dtype=torch.long)
loss(input, target)

Note the documentation also has this torch.long dtype in example code. https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html

#Edit 1 The reason it's not working is because of the way you defined input/target tensors in your code. Use torch.tensor with a small 't' instead of torch.Tensor. For a detailed discussion see What is the difference between torch.tensor and torch.Tensor?.

#this will work. Also notice the decimal. otherwise it will be interpreted differently by pytorch
inputs = torch.tensor([[0.,0.],[0.,1.],[1.,0.],[1.,1.]]).to(device=device)
targets = torch.tensor([0.,1.,1.,0.], dtype=torch.long).to(device=device)
user_65529580
Related