In binary classfication, encoding target varible from yes=1, no=0 gives different results than yes=0, no=1 in XGboosting

user_7064105 0
Viewed 429

I am new to Machine Learning. In a binary classfication problem, we encode/transform target variable like yes=1 and No=0 (directly in dataset) it gives follwoing results

  • Accuracy:95
  • Recall: 90
  • Precision:94
  • F1: 92

but if we encode/transform target variable inversely like yes=0 and No=1(directly in dataset), then it gives these results

  • Accuracy:95
  • Recall:97
  • Precision:94
  • F1:95

I am using XGboost algorithm. All other variables are numeric(positive and negative) Although accuracy is same in both cases but I assume that F1 should also be same in both cases. So why it is giving different results. I know that scikit-learn can handle encoding but why F1 is different in both cases?

xtrain,xtest,ytrain,ytest=train_test_split(X,encoded_Y,test_size=0.3,random_state=100,shuffle=True)
clf_xgb = xgb.XGBClassifier(nthread=1,random_state=100)
clf_xgb.fit(xtrain, ytrain)
xgb_pred = clf_xgb.predict(xtest)
xgb_pred_prb=clf_xgb.predict_proba(xtest)[:,1]
print(confusion_matrix(xgb_pred,ytest))
# [984   57]
# [103 1856]

#Find Accuracy of XGBoost
accuracy_xgb = accuracy_score(ytest,xgb_pred)
print("Accuracy: {}".format(accuracy_xgb)

#Find Recall of XGBoost
recall_xgb = recall_score(ytest,xgb_pred)
recall_xgb

#Find Precision of XGBoost
precision_xgb = precision_score(ytest,xgb_pred)
precision_xgb

#Find F1 Score XGB
xgb_f1=f1_score(ytest,xgb_pred)
xgb_f1
precision-recall xgboost classification machine-learning python
1 Answers

This is because the f1 score and precision and recall are connected.

The formulas are:

f1= 2/((recall^-1)+(precision^-1))

and

recall= true_positives / (true_positives + false_negatives)
precision= true_positives / (true_positives + false_positives)

So recall and precision are depending on what you define as positive (1). If you Switch your positive / negative cases, like you do by mapping yes/no differently, you get a totally different result. You can see that by the following calculation, assuming you have 100 yes and 4900 nos and get the follwoing result:

              ----------
              |YES|NO  |
|-------------|---|----|
|predicted_YES| 90|   5|
|-------------|---|----|
|predicted_NO | 10|4895|
------------------------

Then in case you define YES as positive (1), you get

precision=90/(90+5)=0.947
recall=90/(90+10)=0.9
f1= 2/(precision^(-1) + recall^(-1))=0.923

While if you define NO as positive (1), you get:

precision=4895/(4895+10)=0.998
recall=4895/(4895+10)=0.999
f1= 2 / (precision^(-1) + recall^(-1))=0.998

Note, if YES is your positive class, the matrix above is assigned to true_positives, ... like this:

              --------
              |YES|NO|
|-------------|---|--|
|predicted_YES|TP |FP|
|-------------|---|--|
|predicted_NO |FN |TN|
----------------------

While if you define NO to be the positive class, the true_positives, ... are assigned like this:

              --------
              |YES|NO|
|-------------|---|--|
|predicted_YES|TN |FN|
|-------------|---|--|
|predicted_NO |FP |TP|
----------------------
user_69251850
Related