How can I write an asymmetric loss function for XGBoost?

Viewed 43

I have been following this article to come up with a custom asymmetric loss function that penalises underestimates more than the overestimates:

enter image description here

The 'a' factor in the code for some reason just cannot seem to be tuned. As soon as I try to take it beyond 0.01 it gives me an empty Booster and I get the following result when I try to predict anything:

ValueError: Booster.get_score() results in empty. This maybe caused by having all trees as decision dumps.

Here is what the complete loss function looks like for me:

# Implement a custom loss for XGBoost
from typing import Tuple

def gradient_linex(predt: np.ndarray, dtrain: xgb.DMatrix, a=1e-2)-> np.ndarray:
    #Compute the gradient of linear-exponential loss
    y, predt, a = dtrain.get_label().astype('float128'), predt.astype('float128'), np.float128(a)
    return -(2/a) * ( np.exp(a * (y-predt)) - 1 )

def hessian_linex(predt: np.ndarray, dtrain: xgb.DMatrix, a=1e-2)-> np.ndarray:
    #Compute the hessian of linear-exponential loss
    y, predt, a = dtrain.get_label().astype('float128'), predt.astype('float128'), np.float128(a)
    return 2 * np.exp( a * (y-predt) )

def linex(predt: np.ndarray, dtrain: xgb.DMatrix, a=1e-2) -> Tuple[np.ndarray, np.ndarray]:
    #Linear-exponential loss
    grad = gradient_linex(predt, dtrain, a=a)
    hess = hessian_linex(predt, dtrain, a=a)
    return grad, hess

def get_linex_function(a=1e-2):
    #returns the linex function with fixed parameters a
    return lambda predt, dtrain: linex(predt, dtrain, a=a)

Here is how I am training my model:

a=1e-2

train_xgb = xgb.DMatrix(X_train, label = y_train)
val_xgb= xgb.DMatrix(X_val, label = y_val)

parameters2 = {
              "tree_method":treemethod,
              "gamma":gamma,
              "reg_lambda":lambda_val,
              "max_depth":maxdepth,
              "eta":eta
             }


reg2 = xgb.train(params = parameters2,
                dtrain = train_xgb,
                obj=get_linex_function(a=a),
                evals=[(val_xgb, 'val_xgb')],
                num_boost_round = nestimators,
                early_stopping_rounds=esr,
                verbose_eval=False)

Can someone please help me understand what the problem is here or just help me come up with a completely different solution?

0 Answers
Related