Keras how to force zero gamma initialization for BatchNormalization for downloaded models

Viewed 93

I'd like to change default gamma initialization for BatchNormalization layers for downloaded models. For example: I have a downloaded EfficientNetB0 model:

from tensorflow.python.keras.applications.efficientnet import EfficientNetB0

model = EfficientNetB0(include_top=False, weights=None, input_shape=(200,200,3))

and after after downloading it is already initialized with gamma = 1:

for layer in model.layers:
    if isinstance(layer, BatchNormalization):
        pprint(layer.gamma.numpy())
        break

# prints:
# array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
#        1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
#       dtype=float32)

so code like below won't work:

from tensorflow.keras.layers import BatchNormalization

for layer in model.layers:
    if isinstance(layer, BatchNormalization):
        layer.gamma_initializer = 'zeros'

# layer.gamma.numpy() is still:
# array([1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.,
#        1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1., 1.],
#       dtype=float32)

Is there any way to force zero gamma initialization?

0 Answers
Related