Parallel processing Flux::groupBy hangs

Viewed 22

I'm using reactor 3.4.18 and have a question about Flux.groupBy. I have generated 1000 integers and splited them into 100 groups, I expect that each group could be process in sperate thread but it hangs after several integers processed.

    @Test
    void shouldGroupByKeyAndProcessInParallel() {
        final Scheduler scheduler = Schedulers.newParallel("group", 1000);

        StepVerifier.create(Flux.fromStream(IntStream.range(0, 1000).boxed())
                .groupBy(integer -> integer % 100)
                .flatMap(groupedFlux -> groupedFlux
                        .subscribeOn(scheduler) // this line doesn't help
                        .doOnNext(integer -> log.info("processing {}:{}", groupedFlux.key(), integer)),
                    2)
            )
            .expectNextCount(1000)
            .verifyComplete();
    }

test execution logs:

10:47:58.670 [main] DEBUG reactor.util.Loggers - Using Slf4j logging framework
10:47:58.846 [group-1] INFO com.huawei.hwclouds.coney.spike.FluxGroupByTest - processing 0:0
10:47:58.866 [group-1] INFO com.huawei.hwclouds.coney.spike.FluxGroupByTest - processing 1:1
10:47:58.867 [group-1] INFO com.huawei.hwclouds.coney.spike.FluxGroupByTest - processing 0:100
10:47:58.867 [group-1] INFO com.huawei.hwclouds.coney.spike.FluxGroupByTest - processing 1:101
10:47:58.867 [group-1] INFO com.huawei.hwclouds.coney.spike.FluxGroupByTest - processing 0:200
10:47:58.867 [group-1] INFO com.huawei.hwclouds.coney.spike.FluxGroupByTest - processing 1:201
-------- start hanging ----------

I have changed the flatmap concurrecy to 2 to speed up the reproduction. I expects that flatmap should only slow the whole processing time but should not hang

0 Answers
Related