Why does LightGBM regression give zero SHAP mean values?

user_11109745 0
Viewed 1279

As you can see in SHAP waterfall plot the values are zero, what is the reason for it? Are the zero values reasonable?

enter image description here

Here is the link to my data: https://github.com/kilickursat/Tunnelling/blob/main/TBM_Performance.xlsx

Here is my code:

import numpy as np
import pandas as pd
import lightgbm
from sklearn.metrics import r2_score, mean_squared_error as MSE
from lightgbm import LGBMRegressor
import shap
import io

df2 = pd.read_excel(io.BytesIO(uploaded['TBM_Performance.xlsx'])) #Colab used
df2["ROCK_PRO"] = df2["UCS(MPa)"] / df2["BTS(MPa)"]
X = df2[["UCS(MPa)", "BTS(MPa)","Fs(m)","Alpha(degree)","PI(kN/mm)","ROCK_PRO"]]
y = df2[["ROP(m/hr)"]]
print(df2)
print(X,y)

hyper_params = {
    'task': 'train',
    'boosting_type': 'goss',
    'objective': 'regression',
    'metric': "mse"
}


# train an LightGBM model
model = lightgbm.LGBMRegressor(**hyper_params).fit(X, y)
explainer = shap.Explainer(model)
    
# visualize the first prediction's explanation
shap.plots.waterfall(shap_values[0])
[![enter image description here][2]][2]


from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

X = pd.DataFrame(np.c_[df2['PI(kN/mm)'],df2["ROCK_PRO"],df2["BTS(MPa)"]], columns = ['PI(kN/mm)', "ROCK_PRO", "BTS(MPa)"])
y = df2['ROP(m/hr)']
X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.20, random_state=42)


model= LGBMRegressor(**hyper_params,min_data_in_leaf=0,
min_sum_hessian_in_leaf=0.0).fit(X_train, y_train)

predictions = model.predict(X_test)
r2_score(predictions, y_test).round(2)
#R2_score : 0.96
lightgbm shap machine-learning python
1 Answers

The SHAP values are all zero because your model is returning constant predictions, as all the samples end up in one leaf. This is due to the fact that in your dataset you only have 18 samples, and by default LightGBM requires a minimum of 20 samples in a given leaf (min_data_in_leaf is set to 20 by default). If you set min_data_in_leaf to a smaller value such as 3, then your model will return different predictions for different samples and the SHAP values will be different from zero.

import pandas as pd
from lightgbm import LGBMRegressor
import shap

# import the data
df = pd.read_excel('TBM_Performance.xlsx') 
df['ROCK_PRO'] = df['UCS(MPa)'] / df['BTS(MPa)']
print(df.shape[0])
# 18

# extract the features and target
X = df[['UCS(MPa)', 'BTS(MPa)', 'Fs(m)', 'Alpha(degree)', 'PI(kN/mm)', 'ROCK_PRO']]
y = df[['ROP(m/hr)']]

# train the model with min_data_in_leaf=20
hyper_params = {
    'task': 'train',
    'boosting_type': 'goss',
    'objective': 'regression',
    'metric': 'mse',
}

model = LGBMRegressor(**hyper_params).fit(X, y)
print(model.predict(X))
# [2.52277776 2.52277776 2.52277776 2.52277776 2.52277776 2.52277776
#  2.52277776 2.52277776 2.52277776 2.52277776 2.52277776 2.52277776
#  2.52277776 2.52277776 2.52277776 2.52277776 2.52277776 2.52277776]

# train the model with min_data_in_leaf=3
hyper_params = {
    'task': 'train',
    'boosting_type': 'goss',
    'objective': 'regression',
    'metric': 'mse',
    'min_data_in_leaf': 3,
}

model = LGBMRegressor(**hyper_params).fit(X, y)
print(model.predict(X))
# [2.21428748 2.21428748 2.21428748 2.68171691 2.36794282 2.37986215
#  2.37986215 2.77942405 2.84938042 2.84938042 2.8104722  2.8104722
#  2.50056257 2.47946274 2.46754341 2.58446466 2.58446466 2.24212594]

explainer = shap.Explainer(model)
shap_values = explainer(X)
shap.plots.waterfall(shap_values[0])

enter image description here

user_119890810
Related