I am getting rate limited making too many requests per second to an API.
I am importing contacts into my database and mapping the data to my DB's schema.
The API initially returns its contact 'groups' (ie categories) as an array of links which need to all have separate API calls made in order to retrieve the actual value.
I must iterate over every contact AND make separate API calls for every link in the array associated with each contact. I quickly hit my rate limit (max 10 per second according to API docs) and get a bunch of incomplete entries and errors as a result.
I am using Axios library to make the requests in Node.
My code is below - any help solving this dilemma would be greatly appreciated - thank you!
.then(authUser => {
// RETURN A LIST OF CONTACTS FROM API
axios.get('url', {
headers: {
Authorization: credentials
}
})
.then(response => {
// MAP OVER EACH CONTACT
response.data.map( contact => {
let groups = contact.groupLinks.map(group => {
// MAP OVER EACH LINK IN EACH CONTACTS GROUP ARRAY AND MAKE AJAX REQUEST.
return axios.get(`${group.url}/?alt=json`, {
headers: {
Authorization: 'credentials'
}
})
.then(result => {
return result.data
})
.catch(err => {
console.error('FETCH GROUP MEMBERSHIP FAILURE', err.response.data.error)
next(err)
})
})
return Promise.all(groups)
.then( response => {
return [contact, response]
})