Custom loss function in pytorch 1.10.1

user_4236084 0
Viewed 1328

I am struggeling with defining a custom loss function for pytorch 1.10.1. My model outputs a float ranging from -1 to +1. The target values are floats of arbitrary range. The loss should be a sum of pruducts if the sign between the model output and target is different.

I have searched the internet for quite some hours, but it seems there have been some changes to pytorch throughout the last versions, so I don't really know which example would best fit to my use case and pytorch 1.10.1.

Here is my approach so far:

class Loss(torch.nn.Module):
    @staticmethod
    def forward(self, output, target) -> Tensor:
        loss = 0.0
        
        for i in range(len(target)):
            o = output[i,0]
            t = target[i]
            l = o * t
            if l<0:   #if different sign
                loss -= l

        return loss

Question:

  1. Should I subclass torch.nn.Module or torch.autograd.Function?

  2. Do I need to define @staticmethod?

  3. On some examples, I saw ctx instead of self being used and invocations of ctx.save_for_backward etc. Do I need this? What is its purpose?

  4. When subclassing torch.nn.Module, my code complains: 'Tensor' object has no attribute 'children'. What am I missing?

  5. When subclassing torch.autograd.Function, my code complains about not having a backward function defined. How should my backward function look like?

loss-function pytorch
1 Answers

Custom loss functions can be as simple as a python function. You can simplify this a bit:

def custom_loss(output, target):
    prod = output[:,0]*target
    return -prod[prod<0].sum()
user_104757620
Related