Context: I am trying to pulling data from an API into a warehouse using AWS lambda functions while using python.
The problem: I get connection reset by peer error because i am making multiple API calls in one lambda function and the vendor server says 'slow down buddy'.
This shouldnt be a problem as API responses are super fast, but i am calling 5 endpoints and each response has pagination.
Questions: What is the best practice to avoid throttling issues? E.g i could make 5 lambda functions and 5 API calls, but that makes extra SQL work to join tables.
Let me know what you guys think! Peace