SHAP Exception: Additivity check failed in TreeExplainer

user_12195048 0
Viewed 3006

I am trying to create shap values for a single row for the local explanation but I am consistently getting this error. I tried various methods but still couldn't able to fix them.

Things I did so far -

created the randomized decision tree model -

from sklearn.ensemble import ExtraTreesRegressor
extra_tree = ExtraTreesRegressor(random_state=42)
extra_tree.fit(X_train, y_train)

Then try to calculate the shap values -

# create a explainer object
explainer = shap.Explainer(extra_tree)    
explainer.expected_value
array([15981.25812347])

#calculate shap value for a single row
shap_values = explainer.shap_values(pd.DataFrame(X_train.iloc[9274]).T)

This gives me this error -

Exception: Additivity check failed in TreeExplainer! Please ensure the data matrix you passed to the explainer is the same shape that the model was trained on. If your data shape is correct then please report this on GitHub. Consider retrying with the feature_perturbation='interventional' option. This check failed because for one of the samples the sum of the SHAP values was 25687017588058.968750, while the model output was 106205.580000. If this difference is acceptable you can set check_additivity=False to disable this check.

The shape of training and the single row, I passed has the same number of columns

X_train.shape
(421570, 164)
(pd.DataFrame(X_train.iloc[9274]).T).shape
(1, 164)

And I don't think, it should cause any problem. But to make sure, I also tried to bring the right shape using reshape method.

shap_values = explainer.shap_values(X_train.iloc[9274].values.reshape(1, -1))

X_train.iloc[9274].values.reshape(1, -1).shape
(1, 164)

Which also doesn't solve the problem. So, I thought maybe I also need to match the number of rows. So I created a small data frame and try to test it.

train = pd.concat([X_train, y_train], axis="columns")
train_small = train.sample(n=500, random_state=42)
X_train_small = train_small.drop("Weekly_Sales", axis=1).copy()
y_train_small = train_small["Weekly_Sales"].copy()

# train a randomized decision tree model
from sklearn.ensemble import ExtraTreesRegressor
extra_tree_small = ExtraTreesRegressor(random_state=42)
extra_tree_small.fit(X_train_small, y_train_small)

# create a explainer object
explainer = shap.Explainer(extra_tree_small)
shap_values = explainer.shap_values(X_train_small)

# I also tried to add the y value like this 
shap_values = explainer.shap_values(X_train_small, y_train_small)

But nothing is working.

One of the people on GitHub suggested uninstalling and reinstall shap's latest version from GitHub -

pip install git+https://github.com/slundberg/shap.git

Also tried it still not working.

Does anyone know how to solve this problem?

shap machine-learning python
3 Answers

I'm still not sure why you are transposing or trying to change the shape of your input, as that is not in the examples, but I think the solution below should exemplify using sklearn's ExtraTreeRegressor and using SHAP. Please note, I did not have access to your data so I had to generate my own data.

Note -- I set this to 1000 samples initially so it would run fast. I later set it to 10000 and it ran, just a little slower.

Let me know if you have questions:

# Import statements
import shap, matplotlib.pyplot as plt, pandas as pd, numpy as np
from sklearn.ensemble import ExtraTreesRegressor
from sklearn.datasets import make_regression
from sklearn.model_selection import train_test_split

## Generating data since none was provided
X, y = make_regression(n_samples=1000, n_features=50, n_informative=45, noise=1, random_state=8)
# Convert data to pandas dataframe as in question
X = pd.DataFrame(data=X, columns=["Feature_{}".format(i) for i in range(X.shape[1])])
X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=8, test_size=0.2)

## Creating model per question
extra_tree = ExtraTreesRegressor(random_state=42, verbose=2)
extra_tree.fit(X_train, y_train)

"""
Provided code:
explainer = shap.Explainer(extra_tree) 
"""
# Our Code
explainer = shap.TreeExplainer(extra_tree) 

# Visualize one value
single_shap_value = explainer(X_test.sample(n=1))
shap.summary_plot(single_shap_value, feature_names=X_test.columns, plot_type='bar')
plt.show()

# Visualize all values
shap_values = explainer(X_test)
shap.summary_plot(shap_values, feature_names=X_test.columns)
plt.show()

This produces images like below: enter image description here

enter image description here

I believe your issues are related to my comment which is about your data shape. If you keep things intact, you should be just fine.

Some notes:

python -V
3.8.8
print(sklearn.__version__)
print(shap.__version__)
0.24.1
0.39.0
user_48765610

Try to make a direct call to the explainer

explainer = shap.Explainer(model)
shap_values = explainer(X)

here X is your row.

user_134124180

From my experience this is caused by a few minor predictors missing in the current data set passed to shap_values function, as compared to the training set on which the SHAP-supporting model was trained (i.e. the model the passed to shap.TreeExplainer) was trained on.

This happens routinely with OHE encoder, when you remove a cat variable or two after encoding to binary columns and now the cat var is missing from the dataset.

A quick workaround (which does not affect much the features ranking by SHAP values in the above scenario where OHE is used very sparingly) is to switch off the additivity check, like this:

check_additivity = False
shap_values = explainer.shap_values(X=data_x, y=data_y, check_additivity=check_additivity)
user_99620070
Related