I am training 'bert-base-cased' model for multiclass classification.
Loss is incredibly high, because training data is very noisy and I'm classifying over 700 classes.
I just wanted to see how the training goes.
However precision, recall, f1_score is not changing and I don't know why. (loss is decreasing)
I checked the value in more detail in case the change was too small, but the value was exactly the same.
I'm posting 'compute_metrics' I used.
Let me know if you need more information.
def compute_metrics(pred):
labels = pred.label_ids
preds = pred.predictions.argmax(-1)
precision, recall, f1, _ = precision_recall_fscore_support(labels, preds, average='weighted')
acc = accuracy_score(labels, preds)
return {
'accuracy': acc,
'f1': f1,
'precision': precision,
'recall': recall
}
trainer = Trainer(
model=model,
args=training_ars,
train_dataset=train_dataset,
eval_dataset=eval_dataset,
tokenizer=tokenizer,
compute_metrics=compute_metrics)