SHAP Deep Explainer does not work for LSTM: "Attribute Error: 'Deep' object has no attribute 'masker'"

Viewed 131

We use Keras to construct our LSTM model as follows:

import keras
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM

#make LSTM model architecture
model2 = Sequential()
model2.add(LSTM(100, return_sequences = True))
model2.add(LSTM(50, return_sequences = True))
model2.add(LSTM(10))
model2.add(Dense(1))
model2.compile(loss='mae', optimizer='adam')

The above model is successfully trained and working, and we need the SHAP to explain the output of the LSTM model.

We attempt to use SHAP as follows:

import shap
explainer  = shap.DeepExplainer(model2,x_train_appended)
shap_values = explainer(x_train_appended)

Executing the above 3 lines throws the following error:

In [56]: import shap
...: explainer = shap.DeepExplainer(model2, x_train_appended)
...: shap_values = explainer(x_train_appended)
WARNING:tensorflow:Layers in a Sequential model should only have a single input tensor, but we receive a <class 'list'> input: [<tf.Tensor: shape=(49586, 1, 23), dtype=float32, numpy=
array([[[0.40824828, 0.02564103, 0.03370786, ..., 0.4494382 ,
0.43333334, 0.59210527]],

   [[0.        , 0.06410257, 0.05617978, ..., 0.4494382 ,
     0.43333334, 0.59210527]],

   [[0.5400617 , 0.06410257, 0.06741573, ..., 0.4494382 ,
     0.43333334, 0.59210527]],

   ...,

   [[0.5400617 , 0.01282051, 0.05617978, ..., 0.07865169,
     0.01111111, 0.05263158]],

   [[0.        , 0.02564103, 0.05617978, ..., 0.07865169,
     0.01111111, 0.05263158]],

   [[0.        , 0.02564103, 0.05617978, ..., 0.07865169,
     0.01111111, 0.05263158]]], dtype=float32)>]
Consider rewriting this model with the Functional API.
Traceback (most recent call last):

File "", line 3, in
shap_values = explainer(x_train_appended)

File "/home/kiton/.local/lib/python3.8/site-packages/shap/explainers/_explainer.py", line 207, in call
if issubclass(type(self.masker), maskers.OutputComposite) and len(args)==2:

AttributeError: 'Deep' object has no attribute 'masker'

Did anyone run into a similar issue when using SHAP Deep Explainer? Am I doing something wrong here? Any feedback is appreciated. Thanks a lot for your time and help in advance!

1 Answers

Could this error be related to using keras? Perhaps, building a model with tensorflow or pytorch would solve the problem?

Related