I am working on a conversational AI that used to understand customer requests and make necessary responses. I have used BERT to train my dataset for NLU tasks intent classification and entity detection / slot filling. Model is having accuracy of 96 percent, but after i noticed that i've also a use case that model should classify similar entity and use it for different purposes.
For example when text like "Change the name from Kevin to John", model should able to identify or classify Kevin as the old name and John as the new name entity.
I tried to label different BIO tags for new names and old names on similar text dataset and trained, but model some time classify old name as new name and vice versa.
With regex also, I'm not seeing any rules that classify old name and new name.
So the requirement is model should able to understand the context of the text and classify which one is new name and old name.
Is there any method in NLP or anything to solve this use case?