How to identify the activation function at a specific layer of a loaded keras model?

user_9127141 0
Viewed 1190

After loading a trained model in keras, model.summary() gives the description of the network layers. But it doesn't contain the information about the activation function in the layers.

How to identify which activation function is used in a specific layer?

keras tensorflow python
2 Answers

You need to access .activation attribute of each layer (if it has one). Try this code sample:

for i, layer in enumerate (model.layers):
    print (i, layer)
    try:
        print ("    ",layer.activation)
    except AttributeError:
        print('   no activation attribute')

Output example:

0 <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x000001E72A499C88>
     <function relu at 0x000001E727D9E558>
1 <tensorflow.python.keras.layers.pooling.MaxPooling2D object at 0x000001E72A49C388>
   no activation attribute
2 <tensorflow.python.keras.layers.convolutional.Conv2D object at 0x000001E72A49AB48>
     <function relu at 0x000001E727D9E558>
3 <tensorflow.python.keras.layers.pooling.MaxPooling2D object at 0x000001E72A49A3C8>
   no activation attribute
4 <tensorflow.python.keras.layers.core.Flatten object at 0x000001E72A48CD88>
   no activation attribute
5 <tensorflow.python.keras.layers.core.Dropout object at 0x000001E72A484D88>
   no activation attribute
6 <tensorflow.python.keras.layers.core.Dense object at 0x000001E72A484A48>
     <function softmax at 0x000001E727D95E58>
user_103964690

You could substitute the try-except statement with hasattr.

For example, given a simple VGG16

import tensorflow as tf
from tensorflow.keras.applications.vgg16 import VGG16

vgg = tf.keras.applications.vgg16.VGG16(
    include_top=True,
    weights='imagenet',
    input_shape=(224,224,3),
    classes=1000,
    classifier_activation='softmax'
)

you can print the activation of each layer (together with its name) in this way:

for i,layer in enumerate(vgg.layers):
    if hasattr(layer, "activation"):
        print(layer.name, layer.activation)
    else:
        print(layer.name, 'None')

Which results in:

input_1 None
block1_conv1 <function relu at 0x7f686ba4f290>
block1_conv2 <function relu at 0x7f686ba4f290>
block1_pool None
block2_conv1 <function relu at 0x7f686ba4f290>
block2_conv2 <function relu at 0x7f686ba4f290>
block2_pool None
block3_conv1 <function relu at 0x7f686ba4f290>
block3_conv2 <function relu at 0x7f686ba4f290>
block3_conv3 <function relu at 0x7f686ba4f290>
block3_pool None
block4_conv1 <function relu at 0x7f686ba4f290>
block4_conv2 <function relu at 0x7f686ba4f290>
block4_conv3 <function relu at 0x7f686ba4f290>
block4_pool None
block5_conv1 <function relu at 0x7f686ba4f290>
block5_conv2 <function relu at 0x7f686ba4f290>
block5_conv3 <function relu at 0x7f686ba4f290>
block5_pool None
flatten None
fc1 <function relu at 0x7f686ba4f290>
fc2 <function relu at 0x7f686ba4f290>
predictions <function softmax at 0x7f686ba4a830>
user_103750490
Related