dtype error when fitting Sklearn Pipeline with a TargetEncoder followed by an XGBoost classifier

user_6402486 0
Viewed 625

I am having the following error when trying to fit a pipeline that uses a XGBoost classifier as its final steps:

DataFrame.dtypes for data must be int, float or bool. Did not expect the data types in fields [Categorical Columns here].

I am using the following pipeline, with a TargetEncoder to encode the categorical columns:

numerical_transformer = MinMaxScaler()
categorical_transformer = TargetEncoder()

numerical_cols = X.select_dtypes(include=['float', 'int']).columns
categorical_cols = X.select_dtypes(include='object').columns

preprocessor = make_column_transformer(
    (categorical_transformer, categorical_cols),
    (numerical_transformer, numerical_cols),
    remainder='passthrough')

clf = XGBClassifier(objective= 'binary:logistic')

pipe = make_pipeline(preprocessor, clf)

pipe.fit(X_train,y_train)

The problem is that, apparently, the TargetEncoder sets object to the data types of the encoded categorical columns. An so, XGBoost throws the error.

So, how could I set the data type to, for example, float before it is used by the XGBClassifier object?

xgboost scikit-learn pandas python
1 Answers

I didn't find an explanation of what was the problem, but I will explain what I did to resolve it.

First, I checked if I could find some NaN or None values in the output of the pipeline when discarding the last step:

pipe = Pipeline(my_pipeline.steps[:-1])
X_trans = pipe.fit_transform(X,y)
print(f'nb of nan: {np.isnan(X_trans).sum()}')
print(f'nb of none: {(r == None).sum()}')  
>>> nb of nan: 0
>>> nb of none: 0

So, as there doesn't seem to be a problem with the data, I choose to use a FunctionTransformer, as recommended by Itamar, to force the dtype to float. The pipeline now looks like this:

pipe = make_pipeline(preprocessor, 
                     FunctionTransformer(lambda x: x.astype('float64')), 
                     clf)

And with this, the problem disappeared.

user_64024860
Related