XGBoost fit() custom eval_metric is not called. Why?

user_1229918 0
Viewed 1175

I am passing a custom evaluation metric function to XGB fit(), but the Python API discards callable (i.e., custom) functions. Why? How are we supposed to pass in this function? Or is this functionality not supported by the Python/Scikit API?

model = XGBClassifier(scale_pos_weight=scale_pos_weight)
model.fit(X_train, y_train, eval_metric=xgb_f1)

In sklearn.py (line 755):

def fit(self, X, y, sample_weight=None, base_margin=None,
            eval_set=None, eval_metric=None,
            early_stopping_rounds=None, verbose=True, xgb_model=None,
            sample_weight_eval_set=None, callbacks=None):
...
        feval = eval_metric if callable(eval_metric) else None
        if eval_metric is not None:
            if callable(eval_metric):
                eval_metric = None           # <<<<<<<<<< If eval_metric is callable, it's discarded
            else:
                xgb_options.update({"eval_metric": eval_metric})
xgboost scikit-learn python
1 Answers

If you take a deeper look into the source code your custom evaluation metric is indeed being used, in the code snippet that you posted above if you look at this line: 

feval = eval_metric if callable(eval_metric) else None

your custom evaluation metric (which is a callable) is being assigned to this variable called feval which is then being passed on to the actual method, train, that does your fitting as done here. And now when you look at the docstring of the train method you can find that feval corresponds to Customized evaluation function. Since xgboost uses the term feval for referring to custom evaluation metric and sklearn uses the term eval_metric, the authors of xgboost retained the variable name as it is for ease of understanding and do this conversion internally.

Hops this helps!

user_116513010
Related