Change the spatial input dimension during training

user_7200139 0
Viewed 110

I am training a yolov4 (fully convolutional) in tensorflow 2.3.0.

I would like to change the spatial input shape of the network during training, to further adjust the weights to different scales.

Is this possible?

EDIT: I know of the existence of darknet, but it suffers from some very specific augmentations I use and have implemented in my repo, that is why I ask explicitly for tensorflow.

To be more precisely about what I want to do.

I want to train for several batches at Y1xX1xC then change the input size to Y2xX2xC and train again for several batches and so on.

tensorflow
2 Answers

It is not possible. In the past people trained several networks for different scales but the current state-of-the-art approach is feature pyramids.

https://arxiv.org/pdf/1612.03144.pdf

Another great candidate is to use dilated convolution which can learn long distance dependencies among pixels with varying distance. You can concatenate the outputs of them and the model will then learn which distance is important for which case

https://towardsdatascience.com/review-dilated-convolution-semantic-segmentation-9d5a5bd768f5

user_80105680

It's important to mention which TensorFlow repository you're using. You can definitely achieve this. The idea is to keep the fixed spatial input dimension in a single batch.

But even better approach is to use the darknet repository from AlexeyAB: https://github.com/AlexeyAB/darknet

Just set, random = 1 https://github.com/AlexeyAB/darknet/blob/master/cfg/yolov4.cfg [line 1149]. It will train your network with different spatial dimensions randomly.

One thing you can do is, start your training with AlexeyAB repo with random=1 set, then take the trained weights file to tensorflow for fine-tuning.

user_46220460
Related