How to reset specific layer weights for transfer learning?

user_4406595 0
Viewed 1415

I am looking for a way to re initialize layer's weights in an existing keras pre trained model.

I am using python with keras and need to use transfer learning, I use the following code to load the pre trained keras models

from keras.applications import vgg16, inception_v3, resnet50, mobilenet
vgg_model = vgg16.VGG16(weights='imagenet')

I read that when using a dataset that is very different than the original dataset it might be beneficial to create new layers over the lower level features that we have in the trained net.

I found how to allow fine tuning of parameters and now I am looking for a way to reset a selected layer for it to re train. I know I can create a new model and use layer n-1 as input and add layer n to it, but I am looking for a way to reset the parameters in an existing layer in an existing model.

conv-neural-network keras deep-learning machine-learning neural-network
1 Answers

For whatever reason you may want to re-initalize the weights of a single layer k, here is a general way to do it:

from keras.applications import vgg16
from keras import backend as K

vgg_model = vgg16.VGG16(weights='imagenet')
sess = K.get_session()

initial_weights = vgg_model.get_weights()

from keras.initializers import glorot_uniform  # Or your initializer of choice

k = 30 # say for layer 30
new_weights = [glorot_uniform()(initial_weights[i].shape).eval(session=sess) if i==k else initial_weights[i] for i in range(len(initial_weights))]

vgg_model.set_weights(new_weights)

You can easily verify that initial_weights[k]==new_weights[k] returns an array of False, while initial_weights[i]==new_weights[i] for any other i returns an array of True.

user_46854710
Related