I am focused on trying to maximise the precision of my model and so am looking at using custom metrics. I want to try use f1_score first up, however am struggling to implement the options I am finding here.
What I don't understand is how are the predictions (or yhat) in this example identified.
Simply lifting the code in the above link generates f_score of 0.
I have attempted to generate the predictions in f1 function:
from sklearn.metrics import f1_score
def lgb_f1_score(y_test,X_test):
y_true = y_test
y_hat = np.round(mod_res.predict(X_test))
return 'f1', f1_score(y_true, y_hat), True
However this returns the error
"TypeError: Cannot use Dataset instance for prediction, please use raw data instead"
X_test is however the dataset before it is converted and when I have a model with auc as the metric, I can successfully generate the predictions on it as above ((mod_res.predict(X_test)).
This is being applied here:
mod_res=lgb.train(params,dtrain,
num_boost_round =50,
valid_sets = [dtrain , dtest],
valid_names=["train","test"],
feval=lgb_f1_score, evals_result=evals_result,
early_stopping_rounds =1)
I am sure that I missing something simple but if someone could help me that would be appreciated,
Thanks,
J