How to get Shap values when using sklearn pipeline and TransformedTargetRegressor?

Viewed 63

I am using a sklearn pipeline for a regression task. In the following code, I preprocess numerical and categorical columns and then remove some of the features using VarianceThreshold. Also, I apply standard scaling on objective variables using TransformedTargetRegressor.

import pandas as pd
import shap

from sklearn.datasets import fetch_openml
from sklearn.model_selection import train_test_split, cross_val_predict
from sklearn.ensemble import GradientBoostingRegressor
from sklearn.impute import SimpleImputer
from sklearn.compose import ColumnTransformer
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import OneHotEncoder, MinMaxScaler, StandardScaler
from sklearn.model_selection import GridSearchCV
from sklearn.metrics import r2_score
from sklearn.feature_selection import VarianceThreshold   # Feature selector
from sklearn.compose import TransformedTargetRegressor


# -----------------------------------------------------------------------------
# Data
# -----------------------------------------------------------------------------

# Ames 
X, y = fetch_openml(name="house_prices", as_frame=True, return_X_y=True)

# In this dataset, categorical features have "object" or "non-numerical" data-type. 
numerical_features = X.select_dtypes(include='number').columns.tolist()   # 37
categorical_features = X.select_dtypes(include='object').columns.tolist()   # 43


X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=.3, random_state=0)


# -----------------------------------------------------------------------------
# Data preprocessing
# -----------------------------------------------------------------------------

numerical_preprocessor = Pipeline(steps=[
    ('impute', SimpleImputer(strategy='mean')),
    ('scale', MinMaxScaler())
])


categorical_preprocessor = Pipeline(steps=[
    ('impute', SimpleImputer(strategy='most_frequent')),
    ('one-hot', OneHotEncoder(handle_unknown='ignore', sparse=False))
])
   

preprocessor = ColumnTransformer(transformers=[
    ('number', numerical_preprocessor, numerical_features),
    ('category', categorical_preprocessor, categorical_features)
], 
        verbose_feature_names_out=True,   
)


# -----------------------------------------------------------------------------
# Pipeline
# -----------------------------------------------------------------------------

model = Pipeline(
    [
        ("preprocess", preprocessor),
        ('selector', VarianceThreshold(0.01)),
        ("regressor", GradientBoostingRegressor(random_state=0)),
    ]
)


# Standard scaling of objective variable
model_trans = TransformedTargetRegressor(
        regressor = model, 
        transformer = StandardScaler()
    )

_ = model_trans.fit(X_train, y_train)


print(f"R2 train: {model_trans.score(X_train, y_train):.3f}")   
print(f"R2 test : {model_trans.score(X_test, y_test):.3f}")   

Next, I want to get Shap values and draw Shap plots. For this purpose, I need processed train & test datasets and name of selected features. I am able to get name of selected features and processed data by:

_ = model.fit(X_train, y_train)
selected_featuure_names = model[:-1].get_feature_names_out()

X_train_processed = model[:-1].fit_transform(X_train)
X_test_processed = model[:-1].transform(X_test)


#selected_featuure_names = model_trans.get_feature_names_out()
# AttributeError: 'TransformedTargetRegressor' object has no attribute 'get_feature_names_out'

To draw Shap plots, I use the following code:

explainer = shap.Explainer(model_trans, feature_names=selected_featuure_names)

However, I get the error:

TypeError: The passed model is not callable and cannot be analyzed directly with the given masker!

I can get Shap values by:

explainer = shap.Explainer(model["regressor"], feature_names=selected_featuure_names)

shap_values = explainer(X_test_processed)   

shap.summary_plot(shap_values, X_test_processed)

shap.summary_plot(shap_values, X_test_processed, plot_type="bar")

However, in the above code I have used model without StandardScaling.

Please let me know if you have any idea to solve this issue.

0 Answers
Related