I want to make thousands of calls to an API which is kind of slow -- tens of seconds to get a response. The only limit is that I can make at most one request per second. What's the best way to do this? I think the following code works, but I feel I should be able to make better use of the threading library somehow. I'm using python 3.3
last_job = datetime.now()
for work in work_list:
while (datetime.now()-last_job).total_seconds() < 1 or threading.active_count() >= max_threads:
time.sleep(.1)
threading.Thread(target=work_function, args=[work]).start()
last_job = datetime.now()