Using f1 score as the evaluation metric in light gbm

Viewed 1056

I am focused on trying to maximise the precision of my model and so am looking at using custom metrics. I want to try use f1_score first up, however am struggling to implement the options I am finding here.

What I don't understand is how are the predictions (or yhat) in this example identified.

Simply lifting the code in the above link generates f_score of 0.

I have attempted to generate the predictions in f1 function:

from sklearn.metrics import f1_score    
    def lgb_f1_score(y_test,X_test):  

    y_true = y_test
    y_hat = np.round(mod_res.predict(X_test)) 
    return 'f1', f1_score(y_true, y_hat), True

However this returns the error
"TypeError: Cannot use Dataset instance for prediction, please use raw data instead"

X_test is however the dataset before it is converted and when I have a model with auc as the metric, I can successfully generate the predictions on it as above ((mod_res.predict(X_test)).

This is being applied here:

mod_res=lgb.train(params,dtrain,
              num_boost_round =50,
                         valid_sets = [dtrain , dtest], 
            valid_names=["train","test"],
            feval=lgb_f1_score, evals_result=evals_result,
            early_stopping_rounds =1)

I am sure that I missing something simple but if someone could help me that would be appreciated,

Thanks,
J

0 Answers
Related