How can I script a field in Kibana that matches first 4 digits of a field?

Question

I'm having a steep learning curve with the syntax and my data has PII so I don't know how to describe more.

I need a new field in kibana in the already indexed documents. This field "C" would be a combination of the first 4 digits of a field "A" that contains numbers up to millions and is of type:keyword, and a field "B" that is type:keyword and is some large number. Later I will use this field "C" that is a unique combination, to compare with a list/array of items ( I will insert the list in a query DSL in Kibana, as I need to build some visualizations and reports with the returned documents). I saw that I could use painless to create this new field, but I don't know exactly if I need to use regex and how to.

EDIT:

As requested more info about the mapping with a concrete example.

"fieldA" : {
        "type: "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
           }
          }
        },
"fieldB" : {
        "type: "text",
        "fields" : {
          "keyword" : {
            "type" : "keyword",
            "ignore_above" : 256
           }
          }
        },

Example of values: FieldA = "9876443320134", FieldB = "000000001". I would like to sum the first 4 digits of FieldA and the full content of FieldB. FieldC would result in a value of "9877".

user_8160318 · Answer

The raw query could look like this:

GET combination_index/_search
{
  "script_fields": {
    "a+b": {
      "script": {
        "source": """
          def a = doc['fieldA.keyword'].value;
          def b = doc['fieldB.keyword'].value;
          
          if (a == null || b == null) {
            return null
          }
          
          def parsed_a = new BigInteger(a);
          def parsed_b = new BigInteger(b);
          
          return  new BigInteger(parsed_a.toString().substring(0, 4)) + parsed_b;
        """
      }
    }
  }
}

Note 1: we're parsing the strings into BigInteger b/c of seemingly insufficient Integer.MAX_VALUE.

Note 2: we're first parsing fieldA and only then calling .toString on it again in order to handle the edge case of fieldA starting w/ 0s like 009876443320134. It's assumed that you're looking for 9876, not 98, which be the result of first calling .substring and then parsing.

If you intend to use it in Kibana visualizations, you'll need an index pattern first. Once you've got one, you can proceed as follows:

then put the script in:

click save and the new scripted becomes available in numeric aggregations and queries:

How can I script a field in Kibana that matches first 4 digits of a field?

1 Answers