giving weight for SynonymFilterFactory terms

user_470583 0
Viewed 159

Any way in Solr to give weights to synonyms? (generated by SynonymFilterFactory )

Longer version of the question / some background:

We'd like to give smaller weight for synonyms words/terms injected by SynonymFilterFactory. So exact matches get higher score.

First use case just to give one static weight for all synonyms and if search-time matches through synonyms it'll have a certain (lower) weight than exact match.

Can't find this in documentation.

Is there is a way for Solr to assign weights for terms produced by SynonymFilterFactory?

Any pointers highly appreciated.

PS. Another use case is to fine-tune each synonyms with a particular weight for each particular synonym (i.e. synonyms="synonyms.txt" would have 3 columns and not 2). It seems not currently possible, so perhaps just static weight for all synonyms described above would be possible.

elasticsearch solr lucene
1 Answers

As in most cases with Lucene, the solution is to use multiple fields - one field with synonyms expanded and one without. That way you can decide whether to search with synonyms enabled at all, or you can score hits in the different fields with different weights - and you can adjust those weights based on your query. In Solr you'd used copyField to index the same content into both field, and you can then adjust the weights when using edismax with field^5 field_with_synonyms to score hits without synonyms five times higher than those with synonyms.

If you really want to do it inside one, single field, it'll require far more brittle and custom setup, where you can use payloads attached to each token to manually score each token differently, but this is a more advanced use case and won't fit neatly into all other functionality. It'll solve your PS use case, though. I'd also recommend checking out one of the presentations from Lucene/Solr Revolution about use cases for payload scoring.

Using two fields is the easy way, using payloads is the more flexible, but also more advanced way.

Returns the float value computed from the decoded payloads of the term specified.

The return value is computed using the min, max, or average of the decoded payloads. A special first function can be used instead of the others, to short-circuit term enumeration and return only the decoded payload of the first term.

The field specified must have float or integer payload encoding capability (via DelimitedPayloadTokenFilter or NumericPayloadTokenFilter). If no payload is found for the term, the default value is returned.

payload(field_name,term): default value is 0.0, average function is used.

payload(field_name,term,default_value): default value can be a constant, field name, or another float returning function. average function used.

payload(field_name,term,default_value,function): function values can be min, max, average, or first.

A file used with the DelimitedPayloadTokenFilter is in the format of token|payload and allows you to attach any numeric value as the "payload" for that token.

user_1376500
Related