Collecting results from a list of Futures in java

user_7489502 0
Viewed 1529

I'm trying to use futures to make concurrent api calls. Code:

private void init() throws ExecutionException, InterruptedException {
    Long start = System.currentTimeMillis();
    List<ApiResponse> responses = fetchAllUsingFuture(ids, 3);
    log.info(responses.toString());
    Long finish = System.currentTimeMillis();
    log.info(MessageFormat.format("Process duration: {0} in ms", finish-start));
}

private List<ApiResponse> fetchAllUsingFuture(List<String> ids, int threadCount) throws ExecutionException, InterruptedException {
    ExecutorService executorService = Executors.newFixedThreadPool(threadCount);
    List<List<String>> chunks = Utils.splitToChunks(ids, threadCount);
    List<Future<List<ApiResponse>>> futures = new ArrayList<>();
    chunks.forEach(chunk -> {
        futures.add(wrapFetchInFuture(chunk));
    });
    Future<List<ApiResponse>> resultFuture = executorService.submit(() -> {
        List<ApiResponse> responses = new ArrayList<>();
        futures.forEach(future -> {
            try {
                responses.addAll(future.get());
            } catch (InterruptedException | ExecutionException e) {
                e.printStackTrace();
            }
        });
        return responses;
    });

    executorService.shutdown();
    return resultFuture.get();
}



private Future<List<ApiResponse>> wrapFetchInFuture(List<String> ids) {
    return new FutureTask<>(() -> {
        List<ApiResponse> responses = new ArrayList<>();
        ids.forEach(id -> {
            responses.add(fetchData(id));
        });
        return responses;
    });
}

private ApiResponse fetchData(String id) {
    ResponseEntity<ApiResponse> response = restTemplate.getForEntity(id, ApiResponse.class);
    log.info(MessageFormat.format("Fetching from {0}", id));
    ApiResponse body = response.getBody();
    log.info(MessageFormat.format("Retrieved {0}", body));
    return body;
}

It doesn't execute, the app starts and then just pends. Futures don't get fulfilled. All advices are appreciated. P.S. I'm aware this is much more easily done using CompletableFuture, I was just wondering how to do this with Futures

java
2 Answers

In the original version of the question, you are creating a list of FutureTasks but never send them to the ExecutorService to run them. The tasks never complete, so Future.get blocks forever.

In the updated version of the question, you have put the code that does the waiting into the executor service as a task. The FutureTasks never run, so FutureTask.get will still block forever.

I would suggest you change the code in fetchAllUsingFuture to:

    List<Callable<List<ApiResponse>>> tasks = new ArrayList<>();
    chunks.forEach(chunk -> {
        tasks.add(wrapFetchInCallable(chunk));
    });
    List<Future<List<ApiResponse>>> futures = executorService.invokeAll(tasks);

where wrapFetchInCallable creates a Callable instead of FutureTask:

private static Callable<List<ApiResponse>> wrapFetchInCallable(List<String> ids) {
    return () -> {
        List<ApiResponse> responses = new ArrayList<>();
        ids.forEach(id -> {
            responses.add(fetchData(id));
        });
        return responses;
    };
}
user_3187580

It looks like you are creating a list of FutureTasks but never send them to the ExecutorService to run them. I have implemented ExecutorService with Future Object as below, i hope it helps you:

Service layer:

public List<MovieDTO> searchMoviesParallel(String limit, String offset, String searchPhrase) throws Exception {

        ExecutorService executor = Executors.newFixedThreadPool(1);
        Future<List<MovieDTO>> digitoonResult = executor.submit(new DigitoonSearchTask(limit, offset, searchPhrase));
        List<MovieDTO> movieDTOList = digitoonResult.get();
        executor.shutdown();

        return movieDTOList;
    }

And my Search task(DigitoonSearchTask class) is as below:

public class DigitoonSearchTask implements Callable<List<MovieDTO>> {

    private String limit;
    private String offset;
    private String searchPhrase;

    private final static String digitoonSearchBaseUrl = "http://apitwo.xxx.com/partner/search/?q=";

    public DigitoonSearchTask(String limit, String offset, String searchPhrase) {
        this.limit = limit;
        this.offset = offset;
        this.searchPhrase = searchPhrase;
    }

    @Override
    public List<MovieDTO> call() throws Exception {
        List<MovieDTO> movieDTOList = new ArrayList<>();
        ObjectMapper mapper = new ObjectMapper();
        try {
            String uri = digitoonSearchBaseUrl + URLEncoder.encode(searchPhrase, "utf-8") + "&limit=" + limit + "&offset=" + offset;
            URL url = new URL(uri);
            HttpURLConnection conn = (HttpURLConnection) url.openConnection();
            conn.setRequestMethod("GET");
            conn.setRequestProperty("Accept", "application/json");
            conn.setRequestProperty("authorization", "xxxxxxxxxx");
            if (conn.getResponseCode() != 200) {
                throw new RuntimeException("Failed : HTTP error code : "
                        + conn.getResponseCode());
            }
            BufferedReader br = new BufferedReader(new InputStreamReader(
                    (conn.getInputStream())));

            String output;
            while ((output = br.readLine()) != null) {
                movieDTOList = Arrays.asList(mapper.readValue(output, MovieDTO[].class));
            }
            br.close();
            conn.disconnect();
        } catch (UnknownHostException e) {
            call();
        } catch (MalformedURLException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }
        return movieDTOList;
    }}

consider that now I have just one API and after getting others they can be added as another Search task in service layer by increasing the thread number.

user_71895970
Related