I have a model with 2 inputs, image and metadata. I want to do the Grad_Cam method to see which features are being selected in the image but I can't seem to do it. I use the Gram_cad functions of the keras examples, https://keras.io/examples/vision/grad_cam/. This is my model:
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_8 (InputLayer) [(None, 224, 224, 3) 0
__________________________________________________________________________________________________
resnet50 (Functional) (None, 7, 7, 2048) 23587712 input_8[0][0]
__________________________________________________________________________________________________
input_9 (InputLayer) [(None, 28)] 0
__________________________________________________________________________________________________
global_average_pooling2d_2 (Glo (None, 2048) 0 resnet50[0][0]
__________________________________________________________________________________________________
dense_4 (Dense) (None, 2048) 59392 input_9[0][0]
__________________________________________________________________________________________________
multiply_2 (Multiply) (None, 2048) 0 global_average_pooling2d_2[0][0]
dense_4[0][0]
__________________________________________________________________________________________________
dense_5 (Dense) (None, 8) 16392 multiply_2[0][0]
==================================================================================================
Total params: 23,663,496
Trainable params: 23,610,376
Non-trainable params: 53,120
When I try to run the code below the error *** AssertionError: Could not compute output KerasTensor(type_spec=TensorSpec(shape=(None, 2048), dtype=tf.float32, name=None), name='multiply_2/mul:0', description="created by layer 'multiply_2'") appears.
preds = flat_model.predict([image,metadata])
i = np.argmax(preds[0])
# Generate class activation heatmap
heatmap = make_gradcam_heatmap(image, flat_model, "multiply_2")
If I change the layer name to "resnet50" a different error appears: *** ValueError: Graph disconnected: cannot obtain value for tensor KerasTensor(type_spec=TensorSpec(shape=(None, 224, 224, 3), dtype=tf.float32, name='input_7'), name='input_7', description="created by layer 'input_7'") at layer "conv1_pad". The following previous layers were accessed without issue: []~´
What am I doing wrong? Is it impossible to do this method since it has 2 inputs?