RXJS + Axios stagger network requests

Question

I'm working with an API that has very strict rate limits and I need to send a number of requests to the same endpoint from names in an array. I set up a simple demo project and I tried this (and may variants of):

const pokemon = ['ditto', 'bulbasaur', 'charizard', 'pikachu'];
const obs = pokemon.map((pk, index) => {
  return from(axios.get(`https://pokeapi.co/api/v2/pokemon/${pk}`)).pipe(delay(1000),map(res => {
    return {id: res.data.id, name: res.data.name, height: res.data.height};
  }));
});

concat(obs).subscribe(data => {
  console.log(data);
});

but the Axios.get()'s all fire off when they are created and the concat().subscribe() just logs 4 observables. If I subscribe to the from().pipe() then after a second all 4 logout at once but then I'm subscribing in a subscribe which is poor.

The solution I settled on feels so cumbersome I have to believe there is a better way:

const axios = require('axios');
const { forkJoin, from } = require('rxjs');
const { map } = require('rxjs/operators');

const pokemon = ['ditto', 'bulbasaur', 'charizard', 'pikachu'];

const obs = pokemon.map((pk, index) => {
  return from(new Promise(resolve => setTimeout(async () => {
    const prom = await axios.get(`https://pokeapi.co/api/v2/pokemon/${pk}`);
    resolve(prom);
  }, index*1000))).pipe(map(res => {
    console.log('fetched: ', pk);
    return {id: res.data.id, name: res.data.name, height: res.data.height};
  }))
})

forkJoin(obs).subscribe(data => {
  console.log(data);
});

This delays the creation of the axios.get(), if I run with node --require debugging-aid/network rxjs_axios_delay.js I can see the delayed network requests and the real API I am hitting is happy, but this feels complicated and not very "RXy".

Anyone got anything better?

user_9632621 · Answer

but the Axios.get()'s all fire off when they are created

this highlights a very interesting trait of Promises: they are eager. I think the defer operator can come in handy:

const pokemon = ['ditto', 'bulbasaur', 'charizard', 'pikachu'];

const obs = pokemon.map((pk, index) => {
  return defer(() => axios.get(`https://pokeapi.co/api/v2/pokemon/${pk}`)).pipe(delay(1000),map(res => {
    return {id: res.data.id, name: res.data.name, height: res.data.height};
  }));
});

concat(...obs).subscribe(data => {
  console.log(data);
});

StackBlitz demo.

The cool thing about defer is that it evaluates the given expression(i.e invokes the callback function) when it is being subscribed to. This means you could also do things like these:

let dep$ = of('A');
const src$ = defer(() => dep$);

if (someCondition) {
  dep$ = of('B')
}

// if `someCondition` is true, `dep$` will be `of('B')`
src$.pipe(...).subscribe()

RXJS + Axios stagger network requests

1 Answers