How to train a neural network with a variable output size

user_19264669 0
Viewed 40

I have a working CNN-LSTM model trying to predict keypoints of human bodyparts on videos.

Currently, I have four keypoints as labels right hand, left hand, head and pelvis. The problem is that on some frames I can't see the four parts of the human that I want to label, so by default I set those values to (0,0) (which is a null coordinate).

The problem that I faced was the model taking in account those points and trying to regress on them while being in a sequence.

Thus, I removed the (0,0) points in the loss calculation and the gradient retropropagation and it works much better.

The problem is that the Four points are still predicted, so I am trying to know by any means how to make it predict a variable number of keypoints.

I thought of adding a third parameter (is it visible ?), but it will probably add some complexity and loose the model.

conv-neural-network pytorch python
1 Answers

I think that you'll have to write a custom loss function that computes the loss between points only when the target coordinates are not null.

See PyTorch custom loss function on writing custom losses.

Something like:

def loss(outputs, labels):
    err = 0
    n = 0
    for xo, xt in zip(outputs, labels):
        if xt.values == torch.zeros(2):  # null coord
            continue
        err += torch.nn.functional.mse_loss(xo, xt)
        n += 1
    return (err / n)

This is pseudo-code only! An alternative form which will avoid the loop is to have an explicit binary vector (as suggested by @leleogere) that you can then multiply by the loss on each coordinate before reducing.

user_168797940
Related