I am trying to evaluate the DL model for Binary classification problem [Labels: 0,1] using R0C curve. I am using tensorflow 2.6 and my data are frames extracted from videos.
I tried two methods for computing the ROC but I am getting AUC less than 0.5 which indicates that the tested models are performing random guess. I also tried predict_proba but from what I have read that it has been depreciated in newer tensorflow versions.
My Code:
with open(os.path.join(model_path, 'configs.yaml'), 'r') as fid:
opts = yaml.safe_load(fid)
testing_model = load_model(os.path.join(model_path, 'model.h5'))
testing_data = self.get_data('test', data_test, {**opts['model_opts'], 'batch_size': 1})
testing_results = np.array(testing_data)
testing_results = testing_model.predict(test_data['data'][0],
batch_size=1, verbose=1)
# Computing Performance metrics
acc = accuracy_score(testing_data['data'][1], np.round(testing_results))
f1 = f1_score(testing_data['data'][1], np.round(testing_results))
auc = roc_auc_score(testing_data['data'][1], np.round(testing_results))
roc = roc_curve(testing_data['data'][1], testing_results)
precision = precision_score(testing_data['data'][1], np.round(testing_results))
recall = recall_score(testing_data['data'][1], np.round(testing_results))
pre_recall = precision_recall_curve(testing_data['data'][1], testing_results)
# Method 1
fpr1, tpr1, threshold2 = metrics.roc_curve(testing_data['data'][1], testing_results)
# Method 2
RocCurveDisplay.from_predictions(testing_data['data'][1] ,testing_results, ax=ax_roc, name='DL_Model')
The following shows the classification report indicating bad performance on the models https://i.stack.imgur.com/BrfmL.jpg
What am I doing wrong that is providing with very low auc and low accuracy for positive labels. My aim is to predict as accurately as possible the positive labels indicated by 1.