RxJS: Throttle rate-limited API calls

user_978005 0
Viewed 557

I've been working (and searching) on finding a solution for this for a few days, and while the following works, I can't help but feel there's a more rxjs-y way of doing this.

I have an array of URLs to download from an external source which has rate limiting, no more than say 100 calls per minute. 

const sources = ['a.jpg', 'b.jpg', 'c.jpg'];

timer(0, 1000).pipe(
  switchMap(index => of(sources[index])),
  takeWhile(_ => _ !== undefined),
  switchMap(url => {
    return from(download(url))
  })
).subscribe(
  next => console.log(next),
  err => console.error(err),
  () => console.info('Done')
)

// Pseudo
function download(url) {
  return new Promise((resolve, reject) => {
    resolve('Downloaded ' + url)
  })
}

Seems a bit roundabout and hacky.

What's the best way to iterate through an array to not get locked out for too many request/second?

rxjs javascript
1 Answers

It depends on when you want to make the delay but you can use for example this that guarantees 1s delay:

import { from, of, merge as mergeStatic, timer } from 'rxjs';
import { delay, concatMap, merge, ignoreElements } from 'rxjs/operators';

const sources = ['a.jpg', 'b.jpg', 'c.jpg'];
const mockRequest = s => of(s)

from(sources)
  .pipe(
    concatMap(url => mergeStatic(
      mockRequest(url),
      timer(1000).pipe(ignoreElements())
    ))
  )
  .subscribe(console.log)

Or if you wanted to make the delay after the request completes (that request time + delay) you could just use mockRequest(url).pipe(delay(1000)).

If you want to frist emit the result and then make the dealy you could just switch the order of operators:

from(sources)
  .pipe(
    concatMap(url => mockRequest(url).pipe(
      merge(timer(1000).pipe(ignoreElements()))
    ))
  )
  .subscribe(console.log)

Live demo: https://stackblitz.com/edit/rxjs6-demo-qwdfgp?file=index.ts

user_3107260
Related