Embedding article title with text when finetuning a bert model

Viewed 46

I want to finetune BERT for token classification aimed at markering most similar parts at the article text based on the title of the article. More detailed: I want to feed bert a list of articles with their titles with some text separator/special token, and reward for text markering of the right article spans (somewhat) based on the title. And let's leave out for now, the problem of article length with regard to BERT's inherent 512 token limitation.

Is it feasable, and what is the recommended special token/marker for separating title from article, when feeding it to bert?

0 Answers
Related