ElasticSearch : "copy_to" a nested fields

user_5595377 0
Viewed 241

I try to use the ES "copy_to" attribute to replicate an object field into a nested field, but I got an error despite my multiple tries. Here is my structure :

      ...
      "identifiedBy": {
        "type": "object",
        "properties": {
          "type": {
            "type": "keyword",
            "copy_to": "nested_identifiers.type"
          },
          "value": {
            "type": "text",
            "analyzer": "identifier-analyzer",
            "copy_to": "nested_identifiers.type"
          },
          "note": {
            "type": "text"
          },
          "qualifier": {
            "type": "keyword"
          },
          "source": {
            "type": "keyword",
            "copy_to": "nested_identifiers.type"
          },
          "status": {
            "type": "text"
          }
        }
      },
      "nested_identifiers": {
        "type": "nested",
        "properties": {
          "type": {
            "type": "keyword",
          },
          "value": {
            "type": "text",
            "analyzer": "identifier-analyzer",
          },
          "source": {
            "type": "keyword",
          }
        }
      }
      ...

The mapping error is

java.lang.IllegalArgumentException: Illegal combination of [copy_to] and [nested] 
mappings: [copy_to] may only copy data to the current nested document or any of its 
parents, however one [copy_to] directive is trying to copy data from nested object [null]
to [nested_identifiers]
  • I also try to place the "copy_to" at the "identifiedBy" root level : doesn't work.
  • I also try to use the a "fields" property into "identifiedBy" and "copy_to" this subfield : doesn't work.

Is anyone knows a solution to solve my problem ?

Thanks for your help.

elasticsearch
1 Answers

Tldr;

Because of how Elasticsearch index nested documents. This is not possible ... without updating the mapping.

There is indeed a work around, using include_in_root: true setting.

Else I suggest you pre process you data before indexing it, and during this pre process copy the data over to the nested field. Maybe using an ingest pipeline ?

Ingest Pipeline

PUT /72270706/
{
  "mappings": {
    "properties": {
      "root_type":{
        "type": "keyword"
      },
      "nested_doc":{
        "type": "nested",
        "properties": {
          "nested_type":{
            "type": "keyword"
          }
        }
      }
    }
  }
}

PUT _ingest/pipeline/set_nested_type
{
  "processors": [
    {
      "set": {
        "field": "nested_doc.nested_type",
        "copy_from": "root_type"
      }
    }
  ]
}

POST /72270706/_doc?pipeline=set_nested_type
{
  "root_type": "a type"
}

GET /72270706/_search

Should give you

{
  "took" : 392,
  "timed_out" : false,
  "_shards" : {
...
  },
  "hits" : {
...
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "72270706",
        "_id" : "laOB0YABOgujegeQNA8D",
        "_score" : 1.0,
        "_source" : {
          "root_type" : "a type",
          "nested_doc" : {
            "nested_type" : "a type"
          }
        }
      }
    ]
  }
}

To work around

      ...
      "identifiedBy": {
        "type": "object",
        "properties": {
          "type": {
            "type": "keyword",
            "copy_to": "nested_identifiers.type"
          },
          "value": {
            "type": "text",
            "analyzer": "identifier-analyzer",
            "copy_to": "nested_identifiers.type"
          },
          "note": {
            "type": "text"
          },
          "qualifier": {
            "type": "keyword"
          },
          "source": {
            "type": "keyword",
            "copy_to": "nested_identifiers.type"
          },
          "status": {
            "type": "text"
          }
        }
      },
      "nested_identifiers": {
        "type": "nested",
        "include_in_root": true,
        "properties": {
          "type": {
            "type": "keyword",
          },
          "value": {
            "type": "text",
            "analyzer": "identifier-analyzer",
          },
          "source": {
            "type": "keyword",
          }
        }
      }
      ...

You will need to re index the existing data.

But be aware the copy_to will not copy the information to the nested object. But to another field, that has the same name but is not nested.

user_45349230
Related