Does batch normalization replace 'layers.experimental.preprocessing.Rescaling' in CNN models?

Viewed 77

Does batch normalization replace 'layers.experimental.preprocessing.Rescaling' in CNN models? Or we should first normalize the data and then use BN in the CNN model.

1 Answers

No. Rescaling and batch normalization are different concepts.

You rescale your input data by a fix scale and offset to be in a certain range. So it is a preprocessing step. Furthermore, it does not have trainable values. Often 0 to 1 or -1 to 1 are used.

In normalization, you want to scale your feature maps in such a way, that you have zero mean with unit variance. Batch normalization normalizes the batch at a certain stage within the model, e.g. after a convolution. It's parameters are trainable, usualy denated as gamma and beta. With these parameters, the model can scale the feature, since probably zero mean/unit variance is not the optimal scaling to converge your training objective.

Related