Python sklearn NotFittedError after XGBoost .fit has been called

user_13830654 0
Viewed 1770

I'm trying to make use of sklearn plot_partial_dependence function on a XGBoost fitted model i.e. after calling .fit. But I keep getting the error:

NotFittedError: This XGBRegressor instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.

Here are the steps I'm taking using a dummy data-set.

Complete example with dummy data:

import numpy as np
# dummy dataset
from sklearn.datasets import make_regression
X_train, y_train = make_regression(n_samples = 1000, n_features = 10)


# Import xgboost
import xgboost as xgb

# Initialize the model 
model_xgb_1 = xgb.XGBRegressor(max_depth = 5, 
                               learning_rate = 0.01, 
                               n_estimators = 100, 
                               objective = 'reg:squarederror', 
                               booster = 'gbtree') 

# Fit the model 
# Not assigning to a new variable 
model_xgb_1.fit(X_train, y_train)

# Just to check that .predict can be called and works
# without error 
print(np.sum(model_xgb_1.predict(X_train)))
# the above works ok and prints the output

#This next step throws an error:
from sklearn.inspection import plot_partial_dependence
plot_partial_dependence(model_xgb_1, X_train, [0])

Output:

662.3468

NotFittedError: This XGBRegressor instance is not fitted yet. Call 'fit' with appropriate arguments before using this estimator.

Update

Workaround for the case when booster = 'gblinear'

# CHANGE 1/2: Use booster = 'gblinear'
# as no coef are returned for the case of 'gbtree' 
model_xgb_1 = xgb.XGBRegressor(max_depth = 5, 
                               learning_rate = 0.01, 
                               n_estimators = 100, 
                               objective = 'reg:squarederror', 
                               booster = 'gblinear') 

# Fit the model 
# Not assigning to a new variable 
model_xgb_1.fit(X_train, y_train)

# Just to check that .predict can be called and works
# without error 
print(np.sum(model_xgb_1.predict(X_train)))
# the above works ok and prints the output


#This next step throws an error:
from sklearn.inspection import plot_partial_dependence
plot_partial_dependence(model_xgb_1, X_train, [0])

# CHANGE 2/2
# Add the following:
model_xgb_1.coef__ = model_xgb_1.coef_
model_xgb_1.intercept__ = model_xgb_1.intercept_

# Now call plot_partial_dependence --- It works ok
from sklearn.inspection import plot_partial_dependence
plot_partial_dependence(model_xgb_1, X_train, [0])
scikit-learn machine-learning python
2 Answers

To avoid this error, do not affect a fitted model to a variable.

# Import xgboost
import xgboost as xgb

# Initialize the model 
model_xgb_1 = xgb.XGBRegressor(max_depth = max_depth, 
                               learning_rate = shrinkage, 
                               n_estimators = nTrees, 
                               objective = 'reg:squarederror', 
                               booster = 'gbtree') 

# Fit the model 
model_xgb_1.fit(X_train, y_train)

# Just to check that .predict can be called and works
# without error 
model_xgb_1.predict(X_train)
# the above works ok and prints the output

#This next step throws an error:
from sklearn.inspection import plot_partial_dependence
plot_partial_dependence(model_xgb_1, X_train, [0])
user_114592260 
from sklearn.ensemble import VotingRegressor
XGB_v=VotingRegressor([("reg",XGB)],)
XGB_RMR=PartialDependenceDisplay.from_estimator(
    XGB_v, x_train, features,
    feature_names=["a"],line_kw={"color": "blue"}
)

This will help you address your help.

user_156201010
Related