Java Stream - Exception while Accumulating Double values with Collectors.reducing()

Viewed 97

I have a list of Author objects. And I need to generate a Map<String,Author> associating a key, which consists of concatenated transId and countryCode, with a value representing an aggregated Author object.

public static class Author {
    String transId;
    String countryCode;
    Double currencyUSD;
    Double exccessAmount;
    
    // constructors, getters, setters, etc.
}

Consider the following example:

Author a1 = new Author("1111111", "US", 1000.00, 50.0);
Author a2 = new Author("1111111", "US", 1000.00, 50.0);
Author a3 = new Author("1111111", "US", 1000.00, 1000.00);
Author a4 = new Author("222222", "US", 1000.00, 1000.00);
Author a5 = new Author("222222", "US", 1000.00, 1000.00);
Author a6 = new Author("3333333", "US", 1000.00, 1000.00);
Author a7 = new Author("444444", "US", 1000.00, 1000.00);
        
List<Author> authors = List.of(
    a1, a2, a3, a4, a5, a6, a7
);

Here there are several objects having same transId and countryCode, like a1, a2 and a3. I want to aggregate them by adding currencyUSD values in each group together, and also exccessAmount values.

My code:

public static void prepareList() {

    List<Author> authors = // initializing the list

    Map<String, Author> mapList2 = authors.stream()
        .collect(Collectors.groupingBy(
            authorObj -> authorObj.getTransId() + "_" + authorObj.getCountryCode(),
            Collectors.reducing(
                new Author("", "", 0D, 0D),
                (x, y) -> { 
                    y.setCurrencyUSD(Double.sum(y.getCurrencyUSD(), x.getCurrencyUSD()));
                    y.setExccessAmount(Double.sum(y.getExccessAmount(), x.getExccessAmount())); 
                    return y; 
                })
            ));
        
}

I'm getting a NullPointerException in the collector while generating the map. How can I add a null-check to my code?

Exception in thread "main" java.lang.NullPointerException at Examples.App.lambda$1(App.java:206) at
java.base/java.util.stream.Collectors.lambda$reducing$42(Collectors.java:877) at
java.base/java.util.stream.Collectors.lambda$groupingBy$53(Collectors.java:1136) at
java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) at
java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) at
java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) 
2 Answers

Object.requireNonNull

You can use Object.requireNonNull(T,String) to validate if your references are null. Provided message would help in identifying the culprit. As I've said in the comments, it's hiding somewhere in your data (your real data, not the sample one) among exccessAmount and currencyUSD values.

Map<String, Author> autorById = authors.stream()
    .collect(Collectors.groupingBy(
        authorObj -> authorObj.getTransId() + "_" + authorObj.getCountryCode(),
        Collectors.reducing(
            new Author("", "", 0D, 0D),
            (y, x) -> {
            y.setCurrencyUSD(y.getCurrencyUSD() +  Objects.requireNonNull(x.getCurrencyUSD(),"currencyUSD of is Null" + x));
            y.setExccessAmount(y.getExccessAmount() +  Objects.requireNonNull(x.getExccessAmount(),"exccessAmount of is Null" + x));
            return y;
        }
    )));

Note:

  • This fix only resolves the problem with a NullPointerException, but the code is still BROKEN - read further!

  • As @Holger has pointed out in the comments, static method Double.sum() was introduced with Java 8 to serve as a convenient implementation of the BinaryOperator via method reference Double::sum. In all other cases, plain plus operator + is a more suitable tool. Because a+b is more straightforward than Double.sum(a,b).

In case if you don't want to treat null value of exccessAmount, or currencyUSD as an abnormal case you can use Objects.requireNonNullElse(T,T) instead. It expects a default value as the second argument. But I would not recommend taking this rout, instead you can simply initialize this fields to zero in your class.

Mutable reduction vs. Immutable reduction

There's another issue, obscured by the exception you're getting. Your code is broken due to the way you're performing reduction.

Collectors.reducing is meant for immutable reduction, i.e. for folding operation in which every reduction step results in creation of a new immutable object like Integer, BigDecimal, etc. But you've provided a mutable identity and performing mutation of this object while folding the stream instead of creating a new one.

As the result of that, the identity object would be created only once per thread. This single instance Author would be used for accumulation and reference to it end up in each and every value. Hence, the result would be incorrect.

Also have a look at the Stream API documentation Reduction and MutableReduction. Here's a quote explaining how Stream.reduce() works (the mechanism behind Collectors.reducing() is the same):

The accumulator function takes a partial result and the next element, and produces a new partial result.

But your accumulator doesn't create a new object while producing a partial result. That's the problem. And here's the proof, a map produced with your sample data (all the values are identical because it's literally the same object):

Author(transId=, countryCode=, currencyUSD=7000.0, exccessAmount=5100.0)
Author(transId=, countryCode=, currencyUSD=7000.0, exccessAmount=5100.0)
Author(transId=, countryCode=, currencyUSD=7000.0, exccessAmount=5100.0)
Author(transId=, countryCode=, currencyUSD=7000.0, exccessAmount=5100.0)
Author(transId=, countryCode=, currencyUSD=7000.0, exccessAmount=5100.0)

To address this issue, you need either:

  • generate a new Author while accumulating values with Collectors.reducing (which would be wasteful since object creation has a cost and your objects are mutable);

  • use mutable reduction.

Here's an example how to implement mutable reduction using a custom collector created via Collector.of():

Map<String, Author> autorById = authors.stream()
    .collect(Collectors.groupingBy(
        authorObj -> authorObj.getTransId() + "_" + authorObj.getCountryCode(),
        Collector.of(
            () -> new Author("", "", 0D, 0D),
            (Author result, Author next) -> {
                result.setCurrencyUSD(result.getCurrencyUSD() + Objects.requireNonNull(next.getCurrencyUSD(), "currencyUSD of is Null" + next)));
                result.setExccessAmount(result.getExccessAmount() + Objects.requireNonNull(next.getExccessAmount(), "exccessAmount of is Null" + next)));
            },
            (left, right) -> {
                left.setCurrencyUSD(left.getCurrencyUSD() + right.getCurrencyUSD()));
                left.setExccessAmount(left.getExccessAmount() + right.getExccessAmount()));
                return left;
            })
    ));

This code can be simplified by extracting the logic from the accumulator and combiner into a method merge() that would reside in the Author class, so that inside the collector there would be only a couple of method references.

/**
    * Returns a {@code Collector} which performs a reduction of its
    * input elements under a specified {@code BinaryOperator} using the
    * provided identity.
    *
    * @apiNote
    * The {@code reducing()} collectors are most useful when used in a
    * multi-level reduction, downstream of {@code groupingBy} or
    * {@code partitioningBy}.  To perform a simple reduction on a stream,
    * use {@link Stream#reduce(Object, BinaryOperator)}} instead.
    *
    * @param <T> element type for the input and output of the reduction
    * @param identity the identity value for the reduction (also, the value
    *                 that is returned when there are no input elements)
    * @param op a {@code BinaryOperator<T>} used to reduce the input elements
    * @return a {@code Collector} which implements the reduction operation
    *
    * @see #reducing(BinaryOperator)
    * @see #reducing(Object, Function, BinaryOperator)
    */

I copied the comments on the reducing method. The first param you input is a new Author(). This new Author will be used on the first time reducing, which means your lambda function (x,y)->{...} will be applied with (x=new Author(),y=a1). The x has only null fields, so you could try to set a default value, or null check to avoid a NullPointerException.

Related