I have a csv file with around 4 million records of lat long and I have 400 such files. I want to get state names for all this data so that I can add them to the files having lat long. I'm using python and I tried using Nominatim but it takes many hours to process just one file. Is there any quick way to get the task done using any loop. Note- Lat Long data is mostly from USA. And all 400 file names are in a sequence. Thanks in advance!
1 Answers
Nominatim usage is restricted to about 1 request per second. If you do more, your IP address will get banned.
There existing many solutions but the main question is how much you can afford to spend in time and money to solve your need?
The main operation you need is called reverse geocoding.
A cheap solution is to deploy an instance of Nominatim locally. However, doing so requires quite some time and a recent machine with a large amount of memory, and ideally an SSD drive.
Otherwise, you have API services that you can use: Google Map API, OpenCageData, Here, etc.
In all cases, to resolve States for your 4M records, I suggest sending requests in batch when possible and/or in parallel.
Related