Does Keras calculate gradients for frozen layers?

Viewed 736

I use Keras with tensorflow backend.
Will Keras still calculate gradients for the layers which I set trainable = False?

I haven't observed a speedup for deep networks (like Resnet-50) when I fix a substantial part of the layers. It looks like the gradients are still being calculated for the fixed layers but their values are multiplied on 0. Can anyone tell me for sure is it true?

Here is an example of the small network, where I fix the first layer.

import numpy as np
import keras
import keras.applications.resnet50

x = keras.layers.Input(shape=(5,))
y = keras.layers.Dense(5)(x)

z = keras.layers.Dense(5)(y)
model = keras.models.Model(x, z)
for layer in model.layers[:2]:
    layer.trainable = False

model.compile(optimizer='rmsprop', loss='mse')
print model.summary()

X = np.random.rand(100, 5)

model.fit(X, X, epochs=100)
1 Answers
Related