I am using MongoDB's aggregation pipeline to generate a new collection B containing aggregated results from collection A. To this purpose I use the $out stage. Every time I run the aggregation pipeline, new documents might be added, some might be updated and some removed.
I would now like to have a change stream over the aggregated collection B in order to be notified when the aggregation generates different results from the previous one (i.e. at least one insertion/update/remove).
However, if I use the $out stage, the collection is recreated on every execution and I get a rename and invalidate change and then the stream is closed. I can use start_after with a resume token to open the stream again but I am not notified of the changes (rename and invalidate).
I tried using $merge to avoid recreating the collection. The change stream is working as I expect but I can no longer delete old documents from collection B.
Is there a way to make my use case work (i.e. the result of the aggregation pipeline is the new content of the collection + get change notification for insert/remove/update from previous collection content)?