How to get results weighted by references from ElasticSearch

user_1832489 0
Viewed 118

I have a dataset consisting of Notes referencing other Notes.

{id:"aaa", note: "lorem ipsum", references: ["ccc"]},
{id:"bbb", note: "lorem ipsum", references: ["aaa","ccc"]},
{id:"ccc", note: "lorem ipsum", references: ["bbb"]},

I want elastic search to use the references to weight the results, so in this case if I search for lorem I should get id "ccc" back since it has the most references. According to their docs, their graph solution does exactly this, but I also see examples where they are doing similar things.

But no explanation of how this is mapped to the Index. So my question is: how does one set up an ES index that uses references (indices)?

elasticsearch
3 Answers

Other answers gave some clues, but then @7379490 provided the answer in another channel:

There is no way of doing this directly in ES. There are two possible solutions: pre calculate references and pass them into ES by mapping a new value to the document.

Or aggregate and use aggregation to sort the response:

{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "note": "lorem"
        }
      },
      "aggs": {
      "references":{
        "terms" : {
          "field" : "references.keyword",
          "order": { "_count": "desc" }
        }
      }
    }
  }
}
user_18324890

Function_score can be used to give higher score to documents with more values for reference field

Mapping:

{
  "mappings": {
    "properties": {
      "id": {
        "type": "integer"
      },
      "note": {
        "type": "text"
      },
      "references": {
        "type": "text",
        "fields": {
          "keyword": {
            "type": "keyword"
          }
        }
      }
    }
  }
}

Query:

{
  "query": {
    "function_score": {
      "query": {
        "match": {
          "note": "lorem"
        }
      },
      "functions": [
        {
          "script_score": {
            "script": "_score * doc['references.keyword'].length" --> references length
          }
        }
      ]
    }
  }
}

Result:

 "hits" : [
      {
        "_index" : "index71",
        "_type" : "_doc",
        "_id" : "iJObRHIBXTOhZcHPaYks",
        "_score" : 0.035661265,
        "_source" : {
          "id" : 2,
          "note" : "lorem ipsum",
          "references" : [
            1,
            3
          ]
        }
      },
      {
        "_index" : "index71",
        "_type" : "_doc",
        "_id" : "h5ObRHIBXTOhZcHPYok2",
        "_score" : 0.017830633,
        "_source" : {
          "id" : 1,
          "note" : "lorem ipsum",
          "references" : [
            3
          ]
        }
      },
      {
        "_index" : "index71",
        "_type" : "_doc",
        "_id" : "iZObRHIBXTOhZcHPb4k4",
        "_score" : 0.017830633,
        "_source" : {
          "id" : 3,
          "note" : "lorem ipsum",
          "references" : [
            2
          ]
        }
      }
    ]
user_92793540

If your data contain referencesGiven and you need to search by referencesReceived, I would recommend a 2-way pass:

  1. insert all documents with an empty field for referencesReceived: []
    (or referencesReceivedCount: 0 if that is enough)
  2. for each document, for each item in referencesGiven, update the document receiving the reference
user_11766010
Related