NLLLoss is just a normal negative function?

user_13746281 0
Viewed 333

I'm having trouble understanding nn.NLLLoss().

Since the code below always prints True then what's the difference between the nn.NLLLoss() and using the negative sign (-)?

import torch
while 1:
   b = torch.randn(1)
   print(torch.nn.NLLLoss()(b, torch.tensor([0])) == -b[0])
loss-function pytorch deep-learning machine-learning python
1 Answers

In your case you only have a single output value per batch element and the target is 0. The nn.NLLLoss loss will pick the value of the predicted tensor corresponding to the index contained in the target tensor. Here is a more general example where you have a total of five batch elements each having three logit values:

>>> logits = torch.randn(5, 3, requires_grad=True)
>>> y = torch.tensor([1, 0, 2, 0, 1])
>>> y_hat = torch.softmax(b, -1)

Tensors y and y_hat correspond to the target tensor and estimated distributions respectively. You can implement nn.NLLLoss with the following:

>>> -y_hat[torch.arange(len(y_hat)), y]
tensor([-0.2195, -0.1015, -0.3699, -0.5203, -0.1171], grad_fn=<NegBackward>)

Compared to the built-in function:

>>> F.nll_loss(y_hat, y, reduction='none')
tensor([-0.2195, -0.1015, -0.3699, -0.5203, -0.1171], grad_fn=<NllLossBackward>)

Which is quite different to -y_hat alone.

user_63313690
Related