Python ProcessPoolExecutor How to lazily load processes on submit, instead of eagerly load a number of processes equal to max_workers?

If I create a ProcessPoolExecutor(max_workers=10), but only submit 1 job, it will actually start 10 processes, not 1. The same is true if you instead create a multiprocessing.Pool(processes=10).

I assume it is eagerly preloading a number of processes equal to max_workers in order to minimize the start up time when submit is executed.

How can I disable this behavior, and only start up a process when executor.submit is invoked? Basically I want to lazily load these processes.

You can reproduce this behavior by executing the following code, and watching your process manager:

import multiprocessing
import threading
import time
from datetime import datetime
from concurrent.futures import ProcessPoolExecutor


def task(i):
    print(f'task {i} started at {datetime.now()}')
    time.sleep(10)
    print(f'task {i} ended at {datetime.now()}')


def main():
    with ProcessPoolExecutor(max_workers=10) as executor:
        f = executor.submit(task, i=0)


if __name__ == '__main__':
    multiprocessing.freeze_support()
    main()
    time.sleep(20)

On Windows you can see that this will start 10 processes, instead of 1 process:

Likewise on Unix:

This eager loading behavior also occurs in multiprocessing.Pool:

from multiprocessing import Pool
import time

def main():
    pool = Pool(processes=10)
    time.sleep(1)

if __name__ == '__main__':
    multiprocessing.freeze_support()
    main()