How to randomly initialize layers in pretrained model?

user_11480186 0
Viewed 321

I am using Xception model with pre initialized weights trained on ImageNet as so:

model = keras.applications.Xception(
    weights='imagenet',
    input_shape=(150,150,3)
)

Now I Would like to take specific layer (by its name, using model.get_layer(layerName)) and then reinitialize its weights to completely random one.

What is the simplest way to do so, and if it is even possible?

tensorflow2.0 conv-neural-network keras tensorflow
1 Answers

You could use a reinitialize function like this:

def reinitialize_layer(model, initializer, layer_name):
    layer = model.get_layer(layer_name)    
    layer.set_weights([initializer(shape=w.shape) for w in layer.get_weights()])

Instead of layer_name you could also work with the layer index. You could also extend the function such that it takes a list of layer names, if you like to reinitialize more than one layer.

Usage example:

import keras

model = keras.applications.Xception(
    weights='imagenet',
    input_shape=(299,299,3)
)

# zeros as illustrative example, change to something else
initializer = keras.initializers.Zeros() 

# check pretrained weights
print(model.get_layer("predictions").get_weights())

# change "predictions" to whatever layer name you like to use instead
reinitialize_layer(model, initializer, "predictions") 

# check weights after reinitialization
print(model.get_layer("predictions").get_weights())

model.compile(...)
model.fit(...)
user_112208840
Related