I'd like to change default gamma initialization for BatchNormalization layers for downloaded models.
For example:
I have a downloaded EfficientNetB0 model:
from tensorflow.python.keras.applications.efficientnet import EfficientNetB0
model = EfficientNetB0(include_top=False, weights=None, input_shape=(200,200,3))
and after after downloading it is already initialized with gamma = 1:
for layer in model.layers:
if isinstance(layer, BatchNormalization):
pprint(layer.gamma.numpy())
break
# prints:
# array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
# 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
# dtype=float32)
so code like below won't work:
from tensorflow.keras.layers import BatchNormalization
for layer in model.layers:
if isinstance(layer, BatchNormalization):
layer.gamma_initializer = 'zeros'
# layer.gamma.numpy() is still:
# array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
# 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
# dtype=float32)
Is there any way to force zero gamma initialization?