I was going through the predict method for the entity linker pipe under spacy, and for some reason the score is defined as the following :
scores = prior_probs + sims - (prior_probs*sims)
Link here
Anybody has experience with this / knows where this formula comes from?
Thanks!
It is taken from Entity Linking via Joint Encoding of Types, Descriptions, and Context section 4 equation 2.
I don't feel confident enough though in explaining the formula in detail, on overall the purpose is to combine probability scores for entitiy candidates derived from external knowledge based resources (KB in the paper), which are the prior probabilities, and scores estimated with a sentence encoder, used to encode the mention to link along with its context, sims in the formula because they compute cosine similarity between the encoded mention vector and all entity candidates (which is why this formula is used only if "incl_context" is true).
