Java Streams concat Streams with Supplier followed by distinct (lazy evaluation behavior)

Question

I have a simple code like this

import java.util.function.Supplier;
import java.util.stream.IntStream;
import java.util.stream.Stream;
public class StreamSupplierVersusConcat {
    public static void main(String[] args) {
         final StreamSupplierVersusConcat clazz = new StreamSupplierVersusConcat();
         clazz.doConcat();                
    }

    private void doConcat(){        
         System.out.println(Stream.concat(buildStreamFromRange(0,1000).get()
                ,buildStreamFromRange(1000,2000).get())
                .anyMatch("1"::equals));                
}

    private Supplier>buildStreamFromRange(final int start,final int end){
        return ()->IntStream.range(start, end)
                .mapToObj(i->{
                    System.out.println("index At: "+i);
                    return String.valueOf(i);
                });
    }    
}

I know that concat is lazy so when i run the code i see that is only generating 2 values that's great but knowning that distinct is a stateful operation i thought that putting that method on the Stream pipeline it would the Stream generate all the values and later do the anyMatch method but if i put it like this

    private void doConcat(){        
         System.out.println(Stream.concat(buildStreamFromRange(0,1000).get()
                ,buildStreamFromRange(1000,2000).get())
                .distinct()//ARE ALL THE VALUES GENERATED NOT REQUIRED HERE???
                .anyMatch("1"::equals));                
}

But with the distinct and without it i am getting the same response.

index At: 0
index At: 1
true

What am i missing? I thought that distinct will see consume all the items before anyMatch would see any. Tested on Java 8.

Thanks a lot.

In resume my grasp that I thought that distinct will see consume all the items before anyMatch would see any. is not correct this example explains it.

private void distinctIsNotABlockingCall(){
    final boolean match = Stream.of("0","1","2","3","4","5","6","7","8","8","8","9","9","9","9","9","9","9","9","9","10","10","10","10")
            .peek(a->System.out.println("before: "+a))
            .distinct()//I THOUGHT THAT NOT ANYMATCH WAS CALLED AFTER DISTINCT HANDLE ALL THE ITEMS BUT WAS WRONG.
            .peek(a->System.out.println("after: "+a))
            .anyMatch("10"::equals);
    System.out.println("match? = " + match);                
}

before: 0
after: 0
before: 1
after: 1
before: 2
after: 2
before: 3
after: 3
before: 4
after: 4
before: 5
after: 5
before: 6
after: 6
before: 7
after: 7
before: 8
after: 8
before: 8 distinct working
before: 8 distinct working
before: 9
after: 9 
before: 9 distinct working
before: 9 distinct working
before: 9 distinct working
before: 9 distinct working
before: 9 distinct working
before: 9 distinct working
before: 9 distinct working
before: 9 distinct working
before: 10
after: 10
match? = true

You can see the distinct is received duplicated and non-duplicated values but also anyMatch are receiving those non-duplicated values and distinct and anyMatch are working at the same time thanks a lot.

user_7237884 · Answer

Streams are lazy because intermediate operations are not evaluated unless terminal operation is invoked. check SO answer here

To the best of my understanding Stream Api streams each element until the terminal operation is applied, and then the next element is streamed.

This will also explain the situation here. Element "0" is streamed, terminal operation is not satisfied. Another one needs to be streamed, "1" now, terminal operation .anyMatch("1"::equals)); is satisfied. No need for any more elements to be streamed. Distinct will be invoked in between without any need to change the streamed elements though.

So if you had after "0" another "0" to be streamed it would not reach the terminal operation at all.

 private void doConcat(){        
     
System.out.println(Stream.concat(buildStreamFromRange(0,1000).get()
                ,buildStreamFromRange(1000,2000).get())
                .distinct()
                .peek( e -> System.out.println(e))
                .anyMatch("1"::equals));

Try adding peek and try to stream 2 "0" elements in the start. Only 1 of them will pass the flow and be printed from peek.

Peek is also to be used for debugging purposes and to see how the flow behaves when you are not sure, so use it to your advantage in the future.

Simple example for future readers:

A more simple example where future readers will be able to understand how lazy operators in stream work is the following:

Stream.of("0","0","1","2","3","4")
                .distinct()
                .peek(a->System.out.println("after distinct: "+a))
                .anyMatch("1"::equals);

Will print

after distinct: 0
after distinct: 1

First "0" goes until the terminal operation but does not satisfy it. Another element must be streamed.

Second "0" is filtered through .distinct() and never reaches terminal operation

Since the terminal operation is not satisfied yet, next element is streamed.

"1" goes through terminal operation and satisfies it.

No more elements need to be streamed.

Java Streams concat Streams with Supplier followed by distinct (lazy evaluation behavior)

1 Answers