System-wide mutex in Python on Linux

user_433282 0
Viewed 39262

Is there any easy way to have a system-wide mutex in Python on Linux? By "system-wide", I mean the mutex will be used by a group of Python processes; this is in contrast to a traditional mutex, which is used by a group of threads within the same process.

EDIT: I'm not sure Python's multiprocessing package is what I need. For example, I can execute the following in two different interpreters:

from multiprocessing import Lock
L = Lock()
L.acquire()

When I execute these commands simultaneously in two separate interpreters, I want one of them to hang. Instead, neither hangs; it appears they aren't acquiring the same mutex.

mutex linux python
6 Answers

My answer overlaps with the other answers, but just to add something people can copy-paste, I often do something like this.

class Locker:
    def __enter__ (self):
        self.fp = open("./lockfile.lck")
        fcntl.flock(self.fp.fileno(), fcntl.LOCK_EX)

    def __exit__ (self, _type, value, tb):
        fcntl.flock(self.fp.fileno(), fcntl.LOCK_UN)
        self.fp.close()

And then use it as:

print("waiting for lock")
with Locker():
    print("obtained lock")
    time.sleep(5.0)

To test, do touch lockfile.lck then run the above code in two or more different terminals (from the same directory).

UPDATE: smwikipedia mentioned my solution is unix-specific. I needed a portable version recently and came up with the following, with the idea from a random github project. I'm not sure the seek() calls are needed but they're there because the Windows API locks a specific position in the file. If you're not using the file for anything other than the locking you can probably remove the seeks.

if os.name == "nt":
    import msvcrt

    def portable_lock(fp):
        fp.seek(0)
        msvcrt.locking(fp.fileno(), msvcrt.LK_LOCK, 1)

    def portable_unlock(fp):
        fp.seek(0)
        msvcrt.locking(fp.fileno(), msvcrt.LK_UNLCK, 1)
else:
    import fcntl

    def portable_lock(fp):
        fcntl.flock(fp.fileno(), fcntl.LOCK_EX)

    def portable_unlock(fp):
        fcntl.flock(fp.fileno(), fcntl.LOCK_UN)


class Locker:
    def __enter__(self):
        self.fp = open("./lockfile.lck")
        portable_lock(self.fp)

    def __exit__(self, _type, value, tb):
        portable_unlock(self.fp)
        self.fp.close()
user_46274710

Just to add a one to the list, there's the posix_ipc library, which has a Semaphore class.

A Semaphore with a count of 1 can be used as a Mutex. To complete the threading trio, the SystemEvent library makes use of posix_ipc and provides an Event as well.

I'd also note that this doesn't poll your hard drive either!

user_36016600
Related