CompletableFuture to collect results to hashmap

Viewed 972

I am reading a list of pdf files from DB and parsing them and performing some tasks with them.
When I am reading this list of pdf's , I have seen extracting images from PDF is taking more time and I don't to block my main thread for reading images. So I want to execute extracting images in a separate thread.
I want to read images from one pdf after another, without loading all the pdfs into memory at once (due to memory concern). So I just want 2 threads; one should be main thread (which reads some text from pdf and does some other stuff) and other should be a thread which extracts images and return set of image objects.

One caveat here is that the images from pdfs can be same in their content , so I want to remove duplicate images using checksum or some other means before collecting their result.
I don't want to hold images in memory until all tasks are getting completed, I want to remove duplicates as on when I get the result of one pdf

So the real question is , I need to submit multiple tasks to a thread pool of size 1 and need to remove duplicates as on when I get the result so that I don't need to hold the image in memory for longer time.

Below is the idea of what have I tried.
I have removed unnecessary things from the code like Images and its content and converted code to String based problem.

public static void main(String[] args) throws InterruptedException, ExecutionException {
        Map<String, Integer> uniqueImages = new HashMap<>();
         ExecutorService newFixedThreadPool = Executors.newFixedThreadPool(1);
        List<CompletableFuture<String>> futureList = new ArrayList<>();
        for(int i = 0; i<20000; i++) {
            CompletableFuture<String> obj = CompletableFuture.supplyAsync(()->{
                //Assume lot of duplicates
                return UUID.randomUUID().toString();
            }, newFixedThreadPool).thenApply((x)->{
                if(uniqueImages.containsKey(x)) {
                    int val = uniqueImages.get(x);
                    uniqueImages.put(x, val+1);
                }
                else {
                    uniqueImages.put(x, 1);
                }
                return x;
            });
            futureList.add(obj);
        }
        
        for(CompletableFuture<String> future: futureList) {
            future.get();
        }
        System.out.println(uniqueImages.size());
    }

I'm worried if this code really works or throws ConcurrentModification Exception.

  • Does uniqueImages map really contains unique images and their counts?
  • Are there any hidden issues?
  • Is there better way to solve my use case?
1 Answers

As long as you are only using thread pool of size 1, there is no problem.

However, if there are multiple threads, then the code is indeed not thread-safe and is prone to data race.
We can demonstrate using the following scenario:

Consider this code:

if(uniqueImages.containsKey(x)) {
    int val = uniqueImages.get(x);
    uniqueImages.put(x, val+1);
}
else {
    uniqueImages.put(x, 1);
}

Imagine thread 1 and thread 2 both return the same string and reach at line if(uniqueImages.containsKey(x)) together.
if will return false in both the threads, and uniqueImages.put(x, 1); will be called in both the threads.
Either a ConcurrentModification exception will be thrown or you will get a wrong count (1 instead of 2).

If you plan to use multiple threads, then you must use a ConcurrentHashMap.

Related