How to fine tune BERT to summarize articles

Viewed 222

I'm learning nlp, and as a study project, i'm trying to face the news summarization dataset, using BERT.

The dataset is simple (in the news_summary_more.csv) - it has articles and headlines columns.

What I think, is to use the articles as feature (X), and headlines as target (Y).

My question is how do I measure success in the fine-tuning, so the model will learn?

The model prediction will almost never be the same the the real target (y != ^y), because there are so many options to summarize the articles, and almost 100% of the cases the model will have somewhat different summarization than the real headline.

Example:

Team A won the game against team B

is different from:

Team A is victorious against team B on the game

Although both has pretty much the same meaning.

So how can I test if the model is getting close to the real target? (maybe somehow using MSE)

1 Answers

f1_score would the right measure for such a dataset

Related