Own ExecutorService used to create CompletableFuture does not terminate

I'm trying to use my own ExecutorService to create a set of CompletableFutures to chain several process steps. These steps might throw exceptions.

When they do, it seems to me the thread in the ExecutorService is not released although I'm trying to handle this case.

class Scratch {
    public static void main(String[] args) throws InterruptedException {
        ExecutorService executor = Executors.newFixedThreadPool(10);

        AtomicInteger counter = new AtomicInteger();
        Supplier<?> throwingException = () -> { throw new RuntimeException("throw " + counter.incrementAndGet()); };
        Function<String, CompletableFuture<?>> process =
                url -> CompletableFuture.supplyAsync(throwingException, executor)
                        .exceptionally(Scratch::log);
        var collect = IntStream.range(1, 10).mapToObj(i -> "url" + i)
                .map(process)
                .toArray(CompletableFuture[]::new);
        final CompletableFuture<Void> together = CompletableFuture.allOf(collect);
        System.out.println("joining");
        together.exceptionally(Scratch::log).join();
        System.out.println("finished");
        if (executor.awaitTermination(5, TimeUnit.SECONDS)) {
            System.out.println("exiting cleanly");
        } else {
            System.out.println("not terminated");
        }
        executor.submit(() -> System.out.println("still executing"));
    }
    static <T> T log(Throwable t) {
        System.out.println(t.getMessage());
        return null;
    }
}

Output is

java.lang.RuntimeException: throw 1
joining
java.lang.RuntimeException: throw 2
java.lang.RuntimeException: throw 3
java.lang.RuntimeException: throw 4
java.lang.RuntimeException: throw 5
java.lang.RuntimeException: throw 6
java.lang.RuntimeException: throw 7
java.lang.RuntimeException: throw 8
java.lang.RuntimeException: throw 9
finished
not terminated

The process started by this also isn't terminated (which is how I noticed).

It seems to me this should mean there are no threads left in the ExecutorService at this point, but that doesn't seem to be the case; if we lower the thread pool capacity, it will still run all submitted tasks, and if we add submit another after the failed termination (eg executor.submit(() -> System.out.println("still executing"));), it will get executed.

If we don't pass our own ExecutorService to the CompletableFutre::supplyAsync, the process will terminate as expected.

I also tried other versions of handling the exceptional state (like using together.whenComplete()), but that has the same result.

Why is this happening, and how can I make sure the ExecutorService terminates correctly?

EDIT: I realized that it's not the exception that's causing the problem, this will occur with any task provided to CompletableFuture with your own executor service, which makes total sense given Eugene's reply. I'm changing the question title.