Java Parallel Stream Produce HashMap

user_8183106 0
Viewed 1128

I have the following test, test the integers ranging from 0 to max and if it is validated, construct the pair (vals[i], i). Finally, I want to produce a HashMap which uses vals[i] as key and the value is the list of integers. The code looks like,

IntStream.range(0, max)
   .parallel()
   .filter(i-> sometest(i))
   .mapToObj(i -> new Pair<>(vals[i],i))
   .collect(groupingBy(Pair::getFirst, mapping(Pair::getSecond, toList())));

My question is, is it possible to use parallel streams to speed up the construction of that map?

Thanks.

java-stream java
2 Answers

If you just wanted to know how to better take advantage of parallelism, you could do something like:

ConcurrentMap<Integer, List<Integer>> map = IntStream.range(0, Integer.MAX_VALUE)
    .parallel()
    .filter(i -> i % 2 == 0)
    .boxed()
    .collect(Collectors.groupingByConcurrent(
        i -> i / 3,
        Collectors.mapping(i -> i, Collectors.toList())));

The intermediate creation of Pairs is unnecessary, and groupingByConcurrent accumulates to the new ConcurrentMap in parallel.

Keep in mind that with parallel stream you are stuck with the common ForkJoinPool. For parallelization it's preferable to use something more flexible like an ExecutorService instead of Java Streams.

user_19071860

These are the conditions you must satisfy so that you can perform a Concurrent Reduction, as stated in the Java Documentation about Parallelism:

The Java runtime performs a concurrent reduction if all of the the following are true for a particular pipeline that contains the collect operation:

  • The stream is parallel.
  • The parameter of the collect operation, the collector, has the characteristic Collector.Characteristics.CONCURRENT. To determine the characteristics of a collector, invoke the Collector.characteristics method.
  • Either the stream is unordered, or the collector has the characteristic Collector.Characteristics.UNORDERED. To ensure that the stream is unordered, invoke the BaseStream.unordered operation.

But whether it will speed up your the construction of your Map will depend on others aspects, as mentioned by @Jigar Joshi, including (but not only) :

  • How many elements you have to process
  • How many threads are already being used by your application

Sometimes the overhead of using parallelism (creating and stopping threads, making them communicate and synchronize, ...) is bigger than the gains.

user_79524530
Related