Model.train() with pre-trained weights makes results all 0 while model.eval() is fine

user_16218964 0
Viewed 64

thanks for your attention to this matter.

I want to continue to train a model with its pre-trained weights. When I evaluate this pre-trained model with model.eval(), everything is fine and the model will generate some reasonable results, but when I want to further train this model and set the mode with model.train(), the problem will occur. During the forward loop, all generated results will be zero after the model.train() statement (batchsize = 1).

Any ideas about why this happens?

Thank you very much.

batch-normalization pytorch python
1 Answers

Batch normalization works by normalizing all ativations according to the batch's estimated mean and variance.
What do you expect those values to be when batchsize=1?

Increase your batchsize and see if the problem occurs.

user_17144100
Related