I am trying to implement the Unlikelihood Training loss that was proposed in this research paper: NEURAL TEXT DEGENERATION WITH UNLIKELIHOOD TRAINING. This loss is an updated version of the negative log-likelihood loss (NLLLOSS).
The main idea of this loss is that it avoids unwanted tokens during the training process.

This is my code:
def NLLLoss(logs, targets, c, alpha=0.1):
out = torch.zeros_like(targets, dtype=torch.float)
for i in range(len(targets)):
# out[i] = logs[i][targets[i]] # The original implementation
out[i] = alpha * (1 - logs[i][c[i]]) * logs[i][targets[i]]
return -out.sum()/len(out)
The commented line is the original NLLLoss implementation. This code well, but I was wondering, is this implementation correct?