In elasticsearch 8.3.2 mapping, a field set index:false, but it still searchable

user_19516523 0
Viewed 66

First create an index named "person" by sending PUT request to https://127.0.0.1:9200/person/_mapping, and the response body is

{
    "acknowledged": true,
    "shards_acknowledged": true,
    "index": "person"
}

After that I create a mapping by sending PUT request to https://127.0.0.1:9200/person/_mapping with the request body:

{
    "properties": {
        "name": {
            "type": "text",
            "index": true
        },
        "sex": {
            "type": "keyword",
            "index": true
        },
        "tel": {
            "type": "keyword",
            "index": false
        }
    }
}

the response body is

{
    "acknowledged": true
}

I added a document with the following information

{
    "name": "小明",
    "sex": "男",
    "tel": "12345678910"
}

Then I used field "tel" to search the document: GET https://127.0.0.1:9200/person/_search

{
    "query": {
        "match": {
            "tel": "1"
        }
    }
}

the response body is

{
    "took": 1,
    "timed_out": false,
    "_shards": {
        "total": 1,
        "successful": 1,
        "skipped": 0,
        "failed": 0
    },
    "hits": {
        "total": {
            "value": 0,
            "relation": "eq"
        },
        "max_score": null,
        "hits": []
    }
}

But in lower version such as v7.8.0, such a request will return an error because it was not allowed to search by "tel" whose index is false

elasticsearch mapping lucene indexing
1 Answers

Usually in the Mapping we have two datastructures for a keyword field: an inverted index (which can be disabled with index: false) and the doc_values (which can be disabled with doc_values: false).

The index is a structure that maps values to document id, while doc_values maps doc_id to the term. Both are enabled by default.

The former allow you to perform searches, while the latter allows you to perform aggregation and sort the values.

So, in theory you are right, you should not be able to search. However, if we check the doc_values documentation we can read:

Numeric types, date types, the boolean type, ip type, geo_point type and the keyword type can also be queried when they are not indexed but only have doc values enabled.

So, to have a keyword field that is not searchable (in anyway). You need to disable both index and doc_values:

{
    "properties": {
        "name": {
            "type": "text",
            "index": true
        },
        "sex": {
            "type": "keyword",
            "index": true
        },
        "tel": {
            "type": "keyword",
            "index": false,
            "doc_values": false
        }
    }
}

If you search on this field you will get a "search_phase_execution_exception" with the reason: failed to create query: Cannot search on field [tel] since it is not indexed nor has doc values.

That being said, you have to evaluate if you are ok in disabling both the datastructures for this field or not.

Even if you disable both, nobody prevents you to add a runtime field (you can add it even in the search-request) with the following definition:

emit(params._source['tel'])

To read the value directly from the source. Please be aware of the performance implication of this choice.

user_76893250
Related