I'm building a Spring Boot REST API where I want to process large data that is being read from the file system. My goal is to stream the objects I'm creating when reading from the file system, so that the first of them can get sent to the frontend, before the whole file is read. Right now I got a service class function like this:
List<MyObject> getData(...) {
List<MyObject> result = new ArrayList<>();
for each line of the file {
result.addAll(getLineData(line));
}
return result;
}
And I want to change the function so that it achieves this:
Stream<MyObject> getData(...) {
1. return reference to resultStream
2. send results of each line into the stream
}
How can I achieve this? I was thinking to use Java 8 Stream API Stream.generate() but I'm not sure how I can get this async behavior with it.
But the async behaviour is essential because otherwise, all objects get stored in the memory first which is not good in terms of RAM limitations and big data.
I'd be thankful for any sort of advice!