I am trying to create a heatmap that displays where my CNN is looking in order to classify the image. For the classification tasks, I am deciding between faulty parts and non-faulty parts (hence only Binary classification). I was trying to replicate this code. However, what I saw is that they used the whole Inception network without changing the top layer. My problem now is that I don't know how to correctly connect the layers so that I can use the gradient function to backpropagate the loss from the end of my model (Dense Layer with one neuron) to the last convolutional layer in the inception network ("mixed10"). So far I get an AssertionError with the message of unconnected gradients
The model I trained:
def create_model():
model_inception= InceptionV3(include_top=False, weights='imagenet',input_shape=(299,299,3))
model_inception.trainable=False
model = Sequential()
model.add(model_inception)
model.add(GlobalAveragePooling2D())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(1, activation='sigmoid'))
model.compile(optimizer='rmsprop',
loss='binary_crossentropy', metrics=['accuracy'])
return model
Grad_CAM Code:
layer_name = 'mixed10'
image = np.expand_dims(X_valid[0], 0)
#input layter to inception
input_lay = model.get_layer(index=0).layers[0].input
#Heatmap creatd from ConvLayer
conv_output_lay = model.get_layer(index=0).get_layer(layer_name).output
#Output Layer of the network
output_lay = model.get_layer(index=-1).output
#Connect conv_output with model.input
incept_part_till_conv = Model(input_lay,conv_output_lay)
conv_output = incept_part_till_conv(model.input)
gradModel = Model(
inputs=[model.input],
outputs=[conv_output,
model.output])
# record operations for automatic differentiation
with tf.GradientTape() as tape:
# cast the image tensor to a float-32 data type, pass the
# image through the gradient model, and grab the loss
# associated with the specific class index
inputs = tf.cast(image, tf.float32)
(convOutputs, predictions) = gradModel(inputs)
loss = predictions[:]
# use automatic differentiation to compute the gradients
grads = tape.gradient(loss, convOutputs)
Then I get the error message. If someone could give me some tips on how I can make it work, that would be really great. Thanks!