Why does python's SharedMemory seem to initialize arrays to zeros

user_8980801 0
Viewed 64

I'm initializing a SharedMemory in python to be shared between multiple processes and I've noticed that it always seems to be filled with zeros (which is fine), but I don't understand why this is occurring as the documentation doesn't state there is a default value to fill the memory with.

This is my test code, opened in two seperate power shells, shell 1:

import numpy as np
from multiprocessing.shared_memory import SharedMemory
def get_array_nbytes(rows, cols, dtype):
    array = np.zeros((rows, cols), dtype=dtype)
    nbytes = array.nbytes
    del array
    return nbytes

rows = 10000000
depths_columns = 18
array_sm = SharedMemory(create=True, size=get_array_nbytes(rows, depths_columns, np.float32), name='array_sm')

shell 2:

from multiprocessing.shared_memory import SharedMemory
import numpy as np
array_sm = SharedMemory("depths_array")
array = np.ndarray((rows, 18), dtype=np.float32, buffer=array_sm.buf)

now in the second shell you can follow this up with:

array[0]
array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.,
       0.], dtype=float32)

or

np.where(array != 0)
(array([], dtype=int64), array([], dtype=int64))

Is this behavior always going to be the case or is this a fluke? Is there some sort of undocumented initialization to zero happening in the background?

shared-memory numpy python
1 Answers

This is operating system dependent. Python doesn't initialize the memory - it just takes the virtual memory address offered by the operating system. On posix systems it uses shm_open, while on Windows its CreateFileMapping. On linux and windows, these calls guarantee that memory is initialized to zero.

It would be a security leak to let the application see whatever left over data happens to be in the RAM from the previous user, so it needs to be filled with something. But this isn't a guarantee from python and its possible that some operating systems (embedded OS perhaps) don't do things that way.

user_6420700
Related