Why does concurrent.futures increase vms memory?

Viewed 241

I find that when I use Python's concurrent.futures.ThreadPoolExecutor the vms memory use (as reported by psutil) increases dramatically.

In [1]: import psutil

In [2]: psutil.Process().memory_info().vms / 1e6
Out[2]: 360.636416

In [3]: from concurrent.futures import ThreadPoolExecutor

In [4]: e = ThreadPoolExecutor(20)

In [5]: psutil.Process().memory_info().vms / 1e6
Out[5]: 363.15136

In [6]: futures = e.map(lambda x: x + 1, range(100))

In [7]: psutil.Process().memory_info().vms / 1e6
Out[7]: 1873.580032

In [8]: e.shutdown()

In [9]: psutil.Process().memory_info().vms / 1e6
Out[9]: 1722.51136

This seems to be somewhat proportional to the number of threads.

1 Answers
Related