TL;DR yes, and this might be an XY problem!
If you create a multiprocessing.Manager() and use its methods to create container primitives (.list and .dict), they will already be synchronized and you don't need to deal with the synchronization primitives yourself
from multiprocessing import Manager, Process, freeze_support
def my_function(d, lst):
lst.append([x**2 for x in d.values()])
def main():
with Manager() as manager: # context-managed SyncManager
normal_dict = {'a': 1, 'b': 2}
managed_synchronized_dict = manager.dict(normal_dict)
managed_synchronized_list = manager.list() # used to store results
p = Process(
target=my_function,
args=(managed_synchronized_dict, managed_synchronized_list)
)
p.start()
p.join()
print(managed_synchronized_list)
if __name__ == '__main__':
freeze_support()
main()
% python3 ./test_so_66603485.py
[[1, 4]]
multiprocessing.Array, is also synchronized
BEWARE: proxy objects are not directly comparable to their Python collection equivalents
Note: The proxy types in multiprocessing do nothing to support comparisons by value. So, for instance, we have:
>>> manager.list([1,2,3]) == [1,2,3]
False
One should just use a copy of the referent instead when making comparisons.
Some confusion might come from the section of the multiprocessing docs on Synchronization Primitives, which implies that one should use a Manager to create synchronization primitives, when really the Manager can already do the synchronization for you
Synchronization primitives
Generally synchronization primitives are not as necessary in a multiprocess program as they are in a multithreaded program. See the documentation for threading module.
Note that one can also create synchronization primitives by using a manager object – see Managers.
If you use simply multiprocessing.Manager(), per the docs, it
Returns a started SyncManager object which can be used for sharing objects between processes. The returned manager object corresponds to a spawned child process and has methods which will create shared objects and return corresponding proxies.
From the SyncManager section
Its methods create and return Proxy Objects for a number of commonly used data types to be synchronized across processes. This notably includes shared lists and dictionaries.
This means that you probably have most of what you want already
- manager object with methods for building managed types
- synchronization via Proxy Objects
Finally, to sum up the thread from comments specifically about instances of Lock objects
- there's no inherent way to tell that some named lock is for anything in particular other than meta-information such as its name, comment(s), documentation ..conversely, they're free to be used for whatever synchronization needs you may have
- some useful class/container can be made to manage both the lock and whatever it should be synchronizing -- a normal
multiprocessing.Manager (SyncManager)'s .list and .dict do this, and a variety of other useful constructs exist, such as Pipe and Queue
- one lock can be used to synchronize any number of actions, but having more locks can be a valuable trade-off as they are potentially unnecessarily blocking access to resources
- a variety of synchronization primitives also exist for different purposes
value = my_queue.get() # already synchronized
if not my_lock1.acquire(timeout=5): # False if cannot acquire
raise CustomException("failed to acquire my_lock1 after 5 seconds")
try:
with my_lock2(): # blocks until acquired
some_shared_mutable = some_container_of_mutables[-1]
some_shared_mutable = some_object_with_mutables.get()
if foo(value, some_shared_mutable):
action1(value, some_shared_mutable)
action2(value, some_other_shared_mutable)
finally:
lock2.release()
