Access to intermediate layers in Keras Functional Model

user_10685631 0
Viewed 480

I am using a transfer learning model is a ay very similar to that explained in Chollet's keras Transfer learning guide. To avoid problems with the batch normalization layer, as stated in the guide and many other places, I have to insert the original pretrained base model as a functional model with the training=false option like this:

  inputs = layers.Input(shape=(224,224, 3))
  x = img_augmentation(inputs)
  baseModel = VGG19(weights="imagenet", include_top=False,input_tensor=x)
  x=baseModel(x,training=False)

  # construct the head of the model that will be placed on top of the
  # the base model
  x=Conv2D(32,2)(x)
  headModel = AveragePooling2D(pool_size=(4, 4))(x)
  headModel = Flatten(name="flatten")(headModel)
  headModel = Dense(64, activation="relu")(headModel)
  headModel = Dropout(0.5)(headModel)
  headModel = Dense(3, activation="softmax")(headModel)
  model = Model(inputs, outputs=headModel)

My problem is that I need to use gradcam as in Chollet's gradcam example page. To do this I need access to the basemodel last convolutional layer but when I summarize my model I get:

Model: "model_163"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_3 (InputLayer)         [(None, 224, 224, 3)]     0         
_________________________________________________________________
img_augmentation (Sequential (None, 224, 224, 3)       0         
_________________________________________________________________
vgg19 (Functional)           (None, 7, 7, 512)         20024384  
_________________________________________________________________
conv2d_2 (Conv2D)            (None, 6, 6, 32)          65568     
_________________________________________________________________
average_pooling2d_2 (Average (None, 1, 1, 32)          0         
_________________________________________________________________
flatten (Flatten)            (None, 32)                0         
_________________________________________________________________
dense_4 (Dense)              (None, 64)                2112      
_________________________________________________________________
dropout_2 (Dropout)          (None, 64)                0         
_________________________________________________________________
dense_5 (Dense)              (None, 3)                 195       
=================================================================
Total params: 20,092,259
Trainable params: 67,875
Non-trainable params: 20,024,384
__________________________________________

Thus, the outputs I need are inside one of the vgg19 functional model layers. How can I access this layer without having to remove the training=True option?

keras-layer batch-normalization tensorflow2.0 transfer-learning heatmap
2 Answers

I generally don't like nesting models in models. Although it encourages modularity and introduce nice structure to complex models, TensorFlow gives trouble when you want to do unconventional things (like computing GradCAM or accessing gradients, etc.). I've found it easier to un-nest the model so that you can access the layer that you like easily.

I recently wrote a tutorial to implement GradCAM on TensorFlow 2 for InceptionNet. It should give you enough context to access the required layer.

user_16990750

So as you see the VGG model in your case has type Functional. When you iterate through your compound model's layers you can check for the type of each layer, like this, find the nested Functional model and work with it's layers:

for layer in model.layers:
    if "Functional" == layer.__class__.__name__:
         #here you can iterate and choose the layers of your nested model
         for _layer in layer.layers:
              # your logic with nested model layers
user_51184310
Related