Shared Memory Exiting Independent Process Deletes Mmap

Viewed 96

I'm trying to get shared_memory to work in python 3.8, and have run into the known bug where accessing a shared memory block with an independent process(not a child process) closes the shared memory block. I have seen turicas monkey patch, but am looking for a workaround in standard python. I believe there are other tools to achieve this since process 2 is read-only.

Minimum reproducible example: Process 1:

#process1.py
import numpy as np
from multiprocessing import shared_memory
import time
if __name__ == '__main__':
    arr = np.random.rand(2,2,2)
    shm = shared_memory.SharedMemory(name='foo',create=True, size=arr.nbytes)
    buff = np.ndarray(arr.shape, dtype=arr.dtype, buffer=shm.buf)
    while 1:
        arr = np.random.rand(2, 2, 2)
        buff[:] = arr[:]
        time.sleep(1)

Process 2:

#process2.py
import numpy as np
from multiprocessing import shared_memory
import time
import sys

if __name__ =='__main__':
    existing_shm = shared_memory.SharedMemory(name='foo')
    for _ in range(3):
        buff = np.ndarray((2,2,2), dtype=np.dtype('float64'), buffer=existing_shm.buf)
        print(buff)
        time.sleep(1.5)

    existing_shm.close()

    sys.exit()

Terminal window 1:

python process1.py

Terminal window 2:

python process2.py
[[[0.71427902 0.46214394]
  [0.6427533 ...
## runs as expected, closing with warning:
_tracker.py:216: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

python process2.py
FileNotFoundError: [Errno 2] No such file or directory: '/foo'

Again, this is a known bug, but I am hoping for someone to point me in the direction to re-write this code (likely with something other than shared_memory).

0 Answers
Related