Is BatchNormalizationLayer considered a layer in a neural network? For example, if we say, Resnet50 has 50 layers, does that mean that some of those layers may be batchnormalization layers?
When building models in Keras I considered it as an extra, similar to a dropout layer or when adding an “Activation layer”. But BatchNormalization has trainable parameters, so... I am confused
It really depends on how precise you define what a "layer" is. This may vary for different authors.
For your ResNet example it is pretty clear: In Section 3.4 Implementation you'll a description of the network, there it say's:
We adopt batch normalization (BN) right after each convolution and before activation, [...].
So convolution and batch normalization is considered as a single layer. Figure 3. in the paper shows a picture of ResNet34 where the batch normalization layers are not even explicitly shown and the layers sum up to 34.
So in conclusion, the ResNet paper does not count batch normalization as extra layer.
Further Keras makes it really easy to check those things for many pretrained models, e.g.:
import tensorflow as tf
resnet = tf.keras.applications.ResNet50()
print(resnet.summary())
In DeepLearning literature, an X layer network simply refers to the usage of learnable layers that constitute the representational capacity of the network.
Activation layers, normalization layers (such as NLR, BatchNorm, etc), Downsampling layers (such as Maxpooling, etc) are not considered.
Layers such as CNN, RNN, FC, and the likes that are responsible for the representational capacity of the network are counted.