Usage of KernelExplainer (SHAP tool) for a Pipeline and a multi-class classification

user_4733085 0
Viewed 1404

I have a Pipeline object for a 3-class classification problem. Because most of the examples I find are for binary classifications, I'm finding a bit difficult to perfectly understand how to get the correct plots for each sample.

So, I have this code (pipe_model is my Pipeline object, previously fitted, and X_train and X_test are my train and test data, respectively):

import shap
explainer = shap.KernelExplainer(pipe_model.predict_proba, X_train, link="logit")
shap_values = explainer.shap_values(X_test, nsamples="auto")

I want a plot for each sample in my X_test. I know that somehow explainer.expected_value and shap_values will have 3 parts for each class I'm classifying. So, for example, if I wanted to plot the importance for the first 2 samples of X_test, I was doing this:

shap.force_plot(explainer.expected_value[0], shap_values[0][0,:], X_test.iloc[0,:], link="logit")
shap.force_plot(explainer.expected_value[1], shap_values[1][0,:], X_test.iloc[1,:], link="logit")

I played a bit around with the indexes in expected_value and shap_values and the plots change in a why I cannot explain. Can someone please let me know how I should plot the force_plot for each sample, for each class?

scikit-learn machine-learning python
1 Answers

After exploring a bit, in case some other people have this same question, I think this would be the correct way to do what I was asking:

To show the features which drove the probability for classification in class 0:

shap.force_plot(explainer.expected_value[0], shap_values[0][0,:], X_test.iloc[0,:], link="logit")
shap.force_plot(explainer.expected_value[0], shap_values[0][1,:], X_test.iloc[1,:], link="logit")

To show the features which drove the probability for classification in class 1:

shap.force_plot(explainer.expected_value[1], shap_values[1][0,:], X_test.iloc[0,:], link="logit")
shap.force_plot(explainer.expected_value[1], shap_values[1][1,:], X_test.iloc[1,:], link="logit")

To show the features which drove the probability for classification in class 2:

shap.force_plot(explainer.expected_value[2], shap_values[2][0,:], X_test.iloc[0,:], link="logit")
shap.force_plot(explainer.expected_value[2], shap_values[2][1,:], X_test.iloc[1,:], link="logit")

It would be great if someone could doublecheck this, but in any case this is what I think is correct now.

user_47330850
Related