Explainability with SHAP values on a custom CNN model issues

Question

I am trying to explain the predictions of a trained model using SHAP. I was following the following source code, that works correctly using RESNET50 in an Imagenet DATASET

from tensorflow.keras.applications.resnet50 import ResNet50, 
preprocess_input

import json
import shap
import tensorflow as tf

# load pre-trained model and choose two images to explain
model = ResNet50(weights='imagenet')
def f(X):
    tmp = X.copy()
    print(tmp.shape)
    input()
    preprocess_input(tmp)
    return model(tmp)
X, y = shap.datasets.imagenet50()


# load the ImageNet class names as a vectorized mapping function from ids to names
url = "https://s3.amazonaws.com/deep-learning-models/image-models/imagenet_class_index.json"
with open(shap.datasets.cache(url)) as file:
    class_names = [v[1] for v in json.load(file).values()]
    
print(len(class_names))
print(X.shape)   
input()    

# define a masker that is used to mask out partitions of the input image, this one uses a blurred background
masker = shap.maskers.Image("inpaint_telea", X[0].shape)

# By default the Partition explainer is used for all  partition explainer
explainer = shap.Explainer(f, masker, output_names=class_names)

# here we use 500 evaluations of the underlying model to estimate the SHAP values
shap_values = explainer(X[1:3], max_evals=500, batch_size=50, outputs=shap.Explanation.argsort.flip[:1])
shap.image_plot(shap_values)

This is the exactly example that can be found HERE

Now I have another RESNET50, but trained for another multi-class classification considering 12 classes. I adapted the source code above, but I have issues when running it.

import numpy as np
import tensorflow.keras.models import load_model, preprocess_input
import shap

def f(data_to_explain):
    tmp = data_to_explain.copy()
    preprocess_input(tmp)
    return model(tmp)

reconstructed_model = load_model("my_model")
data_to_explain=np.load("data_to_use.npy")
class_names = ["0", "1", "2", "3", "4", "5", "6", "7", "8", "9", "10", "11"]

# define a masker that is used to mask out partitions of the input image, this one uses a blurred background
masker = shap.maskers.Image("inpaint_telea", data_to_explain[0].shape)

explainer = shap.Explainer(f, masker, output_names=class_names)

# here we use 500 evaluations of the underlying model to estimate the SHAP values
shap_values = explainer(data_to_explain[1:3], max_evals=500, batch_size=50, outputs=shap.Explanation.argsort.flip[:1])
shap.image_plot(shap_values)

the error that I receive is:

 shap_values = explainer(data_to_explain[1:3], max_evals=500, batch_size=50, outputs=shap.Explanation.argsort.flip[:1])
  File "/usr/local/lib/python3.8/dist-packages/shap/explainers/_partition.py", line 135, in __call__
    return super().__call__(
  File "/usr/local/lib/python3.8/dist-packages/shap/explainers/_explainer.py", line 310, in __call__
    sliced_labels = [labels[index_list] for index_list in output_indices]
  File "/usr/local/lib/python3.8/dist-packages/shap/explainers/_explainer.py", line 310, in 
    sliced_labels = [labels[index_list] for index_list in output_indices]
IndexError: index 852 is out of bounds for axis 0 with size 12

The code is basically identical to the previous one, all shapes match and the only difference is the number of classes. So, what could be my problem?

P.s= you can reproduce my error HERE

Explainability with SHAP values on a custom CNN model issues

0 Answers