How to concat several collections of Observables nested in another Observable

Viewed 782

I'm new to RxJs and I am struggling with a complex scenario for an Angular app where I have observables nested in other observables (simplified hereafter).

To be concise, I have:

  • An Observable that returns a stream with a collection of objects SearchActivity => Observable<SearchActivity[]>
  • Each SearchActivity object have an Observable property 'results$' => BehaviorSubject<SearchActivityResult[]>

I try to extract all the results from inside all the activities to one single Observable. So, from the following example (this is not code, but an idea of the data structure):

Observable<SearchActivity[]> => [
 {id: 1, results$: Observable<SearchActivityResult[]> => [1, 2]},
 {id: 2, results$: Observable<SearchActivityResult[]> => [3, 4, 5]} 
 {id: 3, results$: Observable<SearchActivityResult[]> => [6]} 
]

I would like to extract the result to obtain:

Observable<SearchActivityResult[]> => [1, 2, 3, 4, 5, 6] (Edit: the order is not important)

How can this could be achieved using RxJs?

So far I got something like:

// This is not working properly
this.allResults$ = this.displayedActivities$.pipe(
  mergeMap(activities => {
    return concat(...activities?.map(activity => activity.results$ || of([])));
  })
);

But it doesn't seems to work as expected.

I have prepared a StackBiltz with a more complex scenario here

Update 2021-02-23

I created a new StackBlitz where you can interact with the results by clicking on it. The goal is to get a summary of all results for each database (the table at the bottom on the page) and display the amount of positive/negative results across all the search criterions.

TL;DR => Solution

I have just created a new StackBlitz with less complexity in order to isolate the problem. I have included the suggestions made by Mrk Sef and it's working. I have some other errors with the flat operator and a weird circular reference to json but it is more likely cause by the StackBlitz simulator.

3 Answers

I think you have the right idea, but you need a final step to reduce your stream back into a value for you.

What you've written:

this.allResults$ = this.displayedActivities$.pipe(
  mergeMap(activities => 
    concat(...activities?.map(
      activity => activity?.results$ || of([])
    ))
  )
);

has one emission per entry in activities, but you don't want a series of emissions, you'd like an array merging/flattening the results.

You might try something like:

this.allResults$ = this.displayedActivities$.pipe(
  mergeMap(activities => 
    merge(...activities?.map(
      activity => activity?.results$ || of([])
    ))
  ),
  toArray(),
  map(res => res.flat())
);

or

this.allResults$ = this.displayedActivities$.pipe(
  mergeMap(activities => 
    merge(...activities?.map(
      activity => activity?.results$ || of([])
    ))
  ),
  reduce((acc, curr) => [...acc,...curr], [])
);

Update:

You can run this minimal example:

const displayedActivities$ = of([
  {results$: of([1,2,3])},
  {results$: of([10,20,30])},
  {something: "else"},
  {results$: of([100,200,300])}
]);

const allResults$ = displayedActivities$.pipe(
  mergeMap(activities => 
    merge(...activities.filter(
      activity => activity?.results$ != null
    ).map(
      activity => activity.results$
    ))
  ),
  reduce((acc, curr) => [...acc,...curr], [])
).subscribe(console.log);

The output:

[1, 2, 3, 10, 20, 30, 100, 200, 300]

Update # 2

This is how you might handle changing the property result$ observables to long-lived observables. It's much the same as the toArray(), Array.flat() above, only now it uses combineLatest

This is code that should run. It will keep the most recent emission from each observable as part of the final combined output:

Mocking the Settup

Create displayedActivities$ that emits an array of objects, each with a property result$ that emits some number of times (3, in this case) before completing.

/****
 * Pipeable Operator:
 * Takes arrays emitted by the source and spaces out their
 * values by the given interval time in milliseconds
 ****/
function intervalArray<T>(intervalTime = 1000): OperatorFunction<T[], T> {
  return pipe(
    concatMap((v: T[]) =>
      concat(
        ...v.map((value: T) =>
          EMPTY.pipe(
            delay(intervalTime),
            startWith(value)
          )
        )
      )
    )
  );
}

const displayedActivities$ = of([
  { results$: 
    of([
      [1,2,3],
      [4,5,6],
      [7,8,9]
    ]).pipe(intervalArray(250))
  },
  { results$: 
    of([
      [10,20,30],
      [40,50,60],
      [70,80,90]
    ]).pipe(intervalArray(300))
  },
  { something: "else" },
  { results$: 
    of([
      [100,200,300],
      [400,500,600],
      [700,800,900]
    ]).pipe(intervalArray(350))
  }
]);

Combining for allResults$

const allResults$ = displayedActivities$.pipe(
  mergeMap(activities => 
    combineLatest(...activities.filter(
      activity => activity?.results$ != null
    ).map(
      activity => activity.results$.pipe(startWith([]))
    ))
  ),
  map(latest => latest.flat())
).subscribe(console.log);

The output:

[1,2,3]
[1,2,3,10,20,30]
[1,2,3,10,20,30,100,200,300]
[4,5,6,10,20,30,100,200,300]
[4,5,6,40,50,60,100,200,300]
[4,5,6,40,50,60,400,500,600]
[7,8,9,40,50,60,400,500,600]
[7,8,9,70,80,90,400,500,600]
[7,8,9,70,80,90,700,800,900]

You could try to map the observables from the array using Array#map and pass it through a forkJoin to have parallel requests. Which could be mapped from the source observable using a higher order mapping operator like switchMap.

Try the following

Observable<SearchActivity[]>.pipe(
  switchMap((data: any) => 
    forkJoin(data.map(item => 
      item.results$
    ))
  )
).subscribe({
  next: value => {
    console.log(value);
  },
  error: error => {
    // handle error
  }
});

Note:

  1. Observable<SearchActivity[]> here is a placeholder. Replace it with the actual source observable variable.
  2. forkJoin will emit only when all the source observables complete. If you wish to a stream of data replace it either with combineLatest or zip.

Something like this could work for you:

this.allResults$ = this.displayedActivities$.pipe(
  switchMap(activities => 
    merge(...activities.map(a => a.results$))
  ),                                         
  mergeMap(resultsArray => from(resultsArray)),
  scan((acc, cur) => ([...acc, cur]), [])
);

Here's the flow:

  • Use merge to create a single observable that emits whenever any of the activities.results$ emit
  • Use mergeMap / from() to emit array elements individually
  • Use scan to put all emissions into a single array

The function inside of scan is likely too simplistic as it is simply combining ALL emissions into an array with no accounting for multiple emissions that represent the same result. I see you have an id on the returned SearchActivityResult items, so maybe can build a more sophisticated handler function inside of scan that takes that into account.

But hopefully this gets you started.

Here is a working StackBlitz

Related