I have a 5M records in mysql, and my task is:
for each record, transform it to a json object and insert it to mongodb.
my issue is that while transforming it to a json, before i insert the object to mongo I need to call some google api, and google have 50 requests per second (QPS) limit.
How should I make sure I dont exceed this limit and still have the fastest possible execution?
to test out the api in the function that perform the call to google api i'v put Thread.sleep(200) (i know its bad, just for testing purposes), but now I need to run it on 5M records so I need to performance to be as fast as possible
sorry if the answer is too obvious I might dont understand something fundamental, maybe its because im a newbie :\ thanks!