I've been reading up on concurrence and SysV- and POSIX-style shared memory in general and on Python's multiprocessing package in particular. Now I've got a few questions where I my Google-fu hasn't gotten me anywhere so far:
Q1 As far as I understand from this issue in the Python bugtracker, the recently introduced multiprocessing.shared_memory.SharedMemory uses POSIX named shared memory. When it comes to types like RawArray and RawValue defined in multiprocessing.sharedctypes, however, the Python docs leave much to be desired. How are they implemented internally?
Q2 Is the only practical difference between, say, a RawArray('B') and an instance of multiprocessing.shared_memory.SharedMemory that the latter has a name and is also accessible from unrelated (possibly non-Python) processes? Is their performance comparable?
Q3 How exactly are multiprocessing locks implemented in Python? Don't they also need to use some kind of shared memory? The docs say:
Note
Some of this package’s functionality requires a functioning shared semaphore implementation on the host operating system. Without one, the multiprocessing.synchronize module will be disabled, and attempts to import it will result in an ImportError. See bpo-3770 for additional information.
(Note that multiprocessing.Lock actually originates from multiprocessing.synchronize.Lock.) Now according to the linked issue bpo-3770, Lock uses libc's shm_open() which creates a named semaphore. Is this similar to how SharedMemory works?
Update: According to the source code – in particular, these lines: Link 1, link 2 (thank you, bnaecker!), – on non-Windows systems RawArray and RawValue are just implemented using a temporary file which (at least on Linux) is placed in a directory not backed by storage, namely /dev/shm. This then seems very similar to how POSIX shared storage, i.e. SharedMemory works. So I guess I should replace my question Q1 with the following one: Why was shared_memory introduced in the first place then if RawArray is so similar? The aforementioned issue from the beginning mentions performance reasons but I don't understand where that performance difference (at least on Linux) is supposed to come from?