How to plot KNN using SHAP KernelExplainer?

Question

I'm trying to plot Waterfall and Beeswarm plots using K-nearest neighbors by SHAP (SHapley Additive exPlanations).

I have this error:

'numpy.ndarray' object has no attribute 'base_values'

Here is the code:

from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report
knn = KNeighborsClassifier()
knn.fit(X_train, Y_train)

y_pred = knn.predict(X_test)

knn_train_acc = accuracy_score(Y_train, knn.predict(X_train))
knn_test_acc = accuracy_score(Y_test, y_pred)

print(f"Training Accuracy of KNN Model is {knn_train_acc}")
print(f"Test Accuracy of KNN Model is {knn_test_acc}")

shap.initjs()
explainer = shap.KernelExplainer(knn.predict_proba, X_train)
shap_values = explainer.shap_values(X_test)
plt.title("CM1 KNN",fontsize=30)
shap.plots.waterfall(shap_values[0])
shap.plots.beeswarm(shap_values)

However, i have this warning:

Unlike other reduction functions (e.g. `skew`, `kurtosis`), the default behavior of `mode` typically preserves the axis it acts along. In SciPy 1.11.0, this behavior will change: the default value of `keepdims` will become False, the `axis` over which the statistic is taken will be eliminated, and the value None will no longer be accepted. Set `keepdims` to True or False to avoid this warning.
Unlike other reduction functions (e.g. `skew`, `kurtosis`), the default behavior of `mode` typically preserves the axis it acts along. In SciPy 1.11.0, this behavior will change: the default value of `keepdims` will become False, the `axis` over which the statistic is taken will be eliminated, and the value None will no longer be accepted. Set `keepdims` to True or False to avoid this warning.

and this one:

98%|█████████▊| 126/128 [19:02<00:17,  8.52s/it]X does not have valid feature names, but KNeighborsClassifier was fitted with feature names
X does not have valid feature names, but KNeighborsClassifier was fitted with feature names
 99%|█████████▉| 127/128 [19:11<00:08,  8.76s/it]X does not have valid feature names, but KNeighborsClassifier was fitted with feature names
X does not have valid feature names, but KNeighborsClassifier was fitted with feature names
100%|██████████| 128/128 [19:20<00:00,  9.06s/it]

Here is the output error:

AttributeError                            Traceback (most recent call last)
Input In [6], in ()
     16 shap_values = explainer.shap_values(X_test)
     17 plt.title("CM1 KNN",fontsize=30)
---> 18 shap.plots.waterfall(shap_values[0])
     19 shap.plots.beeswarm(shap_values)

File D:\Newfolder\envs\foo\lib\site-packages\shap\plots\_waterfall.py:45, in waterfall(shap_values, max_display, show)
     42 if show is False:
     43     plt.ioff()
---> 45 base_values = shap_values.base_values
     46 features = shap_values.display_data if shap_values.display_data is not None else shap_values.data
     47 feature_names = shap_values.feature_names

AttributeError: 'numpy.ndarray' object has no attribute 'base_values'

Moreover, all libraries are imported and I have the same issue with support vector machine plots as well. So, how to solve this problem and get the plots?

user_4317058 · Answer

Try the following:

from sklearn.neighbors import KNeighborsClassifier
import shap
from shap import Explanation, KernelExplainer
from shap.plots import waterfall, beeswarm

X, y = shap.datasets.iris(display=True)

knn = KNeighborsClassifier()
knn.fit(X.values, y)
explainer = shap.KernelExplainer(knn.predict_proba, X)
sv = explainer.shap_values(X)
exp = Explanation(sv[0], explainer.expected_value[0], X, feature_names=X.columns)
plt.title("CM1 KNN",fontsize=30)

idx = 0
waterfall(exp[idx])

print(knn.predict_proba(X.iloc[[idx]].values))

[[1. 0. 0.]]

A side note: asking with a reproducible example never hurts.

How to plot KNN using SHAP KernelExplainer?

1 Answers