Elasticsearch Shape query to find the bounding polygon and background color from Nested JSON (Google OCR response)

As you're using bounding box coordinates of words over an image i would suggest you to use shape datatype as there is not limitation of coordinate values.

"vertices": {
            "type": "shape"
        }

Also make sure to manipulate the bounding box coordinates in below format.

[[x1,y1],[x2,y2],[x3,y3],[x4,y4],[x1,y1]]

You might want to change the schema in elasticsearch as it will be easier for you to search different fields in a document.

Post data into document like:

POST /example/_doc
{
    "vertices" : {
    "type" : "polygon",
    "coordinates" : [
        [ [1000.0, -1001.0], [1001.0, -1001.0], [1001.0, -1000.0], [1000.0, -1000.0], [1000.0, -1001.0] ]
    ]
  }
}

For Searching you can use make use of envelope type query so you wouldn't have to write all the coordinates of bounding box, you can set a envelope(rectangle) to be searched in and it will give you all the documents that are contained in the envelope. Note : coordinate input for envelope type searching in a bit different it takes [[minX,maxY],[maxX,minY]] this type of format.

Example:

{
  "query":{
      "bool": {
          "must": {
              "match_all": {}
          },
          "filter": {
              "shape": {
                  "prefix.vertices": {
                      "shape": {
                          "type": "envelope",
                          "coordinates": [
                                  [40,60],
                                  [100,40]
                              ]
                      }
                  }
              }
          }
      }
   }
}

As each field in elasticsearch can have single or multiple(in form of array) values, bgColor can be searched using must-match query for matching all the values to bgColor elements.

Example:

{
  "query": {
      "bool": {
          "must": [
            {"match": {"prefix.wordInfo.bgColor": 1}},
            {"match": {"prefix.wordInfo.bgColor": 2}},
            {"match": {"prefix.wordInfo.bgColor": 3}}
          ]
      }
  }
}

I hope this helps.

geo_shape is intended for ... geographic shapes. So you'll have to either

• normalize your coordinates so that they fit into the lat/long spec limits and only then sync them as such
• or you can treat them as being in the, say, Mercator (EPSG:900913) projection which supports values like [589, 93] and then convert them to actual lat/longs. Though you should keep in mind that, say, the geopoint [0 (lon), 1 (lat)] converted to mercator coords is [0, ~111325]. So you'll also need some scaling factor in order to not make your geopolygon too small. In other words, a difference of 1 mercator coordinate is 0.000009° in WGS 84 which means that your polygons could be too small to index and/or too high-res to effectively store.

Step-by-step guide:

1. Convert and/or normalize your coordinates
2. Convert your [x,y] pairs into [lon, lat] pairs
3. Convert your [r,g,b] colors into rgb strings ([0,0,0] => "#000000") -- unless you intend to query by the actual r,g,b channels
4. Create an index w/ a geoshape mapping:

PUT /example
{
    "mappings": {
        "properties": {
            "location": {
                "type": "geo_shape"
            },
            "bgColor": {
                "type": "keyword"
            }
        }
    }
}

5. Make sure your polygon coordinates start and end with the same geopoint & wrap them in 2 more empty arrays to comply with the geojson standard.
6. Sync your polygons + bgColors

POST /example/_doc
{
    "location" : {
        "type" : "polygon",
        "coordinates" : [
            [ [100.0, 0.0], [101.0, 0.0], [101.0, 1.0], [100.0, 1.0], [100.0, 0.0] ]
        ]
    },
    "bgColor": "#000000"
}

After that there's plenty of resources already answered elsewhere as to how polygons are searched and how to search (color) strings.