How to store country/state/city information in Elasticsearch

Viewed 639

How to store country/state/city information in Elasticsearch

i.e there are many countries
each country has many states
each state has many cities

Its easier to store in a relational database, but if I want to store all possible combinations how should I do this in Elasticsearch

I want to store the country, state, city location in a certain index containing user information

i.e users (first_name, last_name, country, state, city ...)
2 Answers

Please don't confuse Elasticsearch with RDBMS and as you have not mentioned what is your use-case ie its full-text search or aggregation, I will show you how to achieve the full-text search with your data and its easy to achieve and don't require much configuration/complexity to achieve it.

As one user at a time can stay only in one city, state, and country but still if you want to store multiple options for the users that also can be done, you just have to index the , separated values.

If you need the aggregations on these fields then please index these fields as keyword so that you can do aggregations on it.

Complete example for full-text search

Index mapping

{
  "mappings" :{
      "properties" :{
          "first_name" :{
              "type" : "text"
          },
          "last_name" :{
              "type" : "text"
          },
          "country" :{
              "type" : "text"
          },
          "state" :{
              "type" : "text"
          },
          "city" :{
              "type" : "text"
          }
      }
  }
}

Index sample docs

{
  "first_name" : "abc",
  "last_name" : "xyz",
  "country": "USA",
  "state" : "California",
  "city" : "SF"
}
{
  "first_name" : "opster",
  "last_name" : "ninja",
  "country": "Israel",
  "state" : "na",
  "city" : "tel aviv"
}
{
  "first_name" : "abc",
  "last_name" : "xyz",
  "country": "USA",
  "state" : "California, washintion", // not two state
  "city" : "SF"
}

Now search for California will return first and third docs as shown below

{
    "query": {
        "match": {
            "state": "california"
        }
    }
}

And search results

 "hits": [
            {
                "_index": "so_63601020",
                "_type": "_doc",
                "_id": "3",
                "_score": 0.38845783,
                "_source": {
                    "first_name": "abc",
                    "last_name": "xyz",
                    "country": "USA",
                    "state": "California",
                    "city": "SF"
                }
            },
            {
                "_index": "so_63601020",
                "_type": "_doc",
                "_id": "2",
                "_score": 0.2863813,
                "_source": {
                    "first_name": "foo",
                    "last_name": "bar",
                    "country": "USA",
                    "state": "California, washington",
                    "city": "SF"
                }
            }
        ]

Data duplication is a trade-off of any NoSQL database/service, including ElasticSearch. The main idea here is, you don't need a separate index with all the possible city/state/country combinations.

With that being said, each user will probably reside in only one city. Plus it's highly improbable that a person's city/state/country will often change. When it does, you'll simply update that user's document. So frequent updates (an important downside of NoSQL) won't be of any significant concern here.

What I would recommend is making sure you either set fielddata: true and/or make your city/state/country fields also of the keyword data type so you can swiftly answer questions such as 'In which state do most of my users reside?' etc by employing aggregations.

Important: do think about some normalization/standarization, esp. when it comes to cities and states. If we talk about the U.S.:

  • Some people may enter their state as Massachusetts, others as Mass., and most as MA. How are you going to handle that?
  • Similarly, NYC is different from New York which is different from New York City and also new york city.

What's usually done is an address autocomplete/dropdown (both open source & paid services are available) which will provide you with a decent amount of standardization so you can save user info directly in your elasticsearch index. There's also the geocoding aspect but that's a different conversation.

Related