How to synchronize file I/O of many independent applications on Linux?

user_1818608 0
Viewed 310

I'm writing a software for Linux which would actively work with user's files in background concurrently with other applications that I don't control. I want to make my background application to not overwrite changes made by other applications. But there is a problem - unlike Windows Linux doesn't provide mandatory file locking capability which creates possibility of ruining user's work due to race conditions which I'd like to avoid.

So I wonder - are there file-systems available on Linux that provide some kind of synchronization mechanisms such as compare-and-swap operation, all-or-nothing transactions, mandatory file locking (like in Windows)?

concurrency io filesystems transactions linux
4 Answers

I believe there are three possible solutions

1) Make all programs to use a custom file I/O library that implement the features that you need. This solution may not be feasible if you do not have access to the source code. You may also consider to use mmap so that changes are written to memory. You use a background process to synchronize dirty pages to existing or new files.

2) Replace standard C/C++ libraries (such as libc.so) that affected programs would use. You could use ldd to find out library dependency. You need to update source code for standard C/C++ to implement features that you need. This may be too difficult for most people.

3) Create your file system. You may refer to many articles in the internet, such as https://kukuruku.co/post/writing-a-file-system-in-linux-kernel/. This is the best and cleanest solution.

Hope it helps.

user_27451400

Rename is atomic. It is up to your application to compare "eTags" of source and destination (possibly under appropriate locks) before deciding on calling rename().

user_3190630

mmap seems to have this kind of protection your looking for: https://www.kernel.org/doc/html/v4.13/media/uapi/v4l/func-mmap.html

prot The prot argument describes the desired memory protection. Regardless of the device type and the direction of data exchange it should be set to PROT_READ | PROT_WRITE, permitting read and write access to image buffers. Drivers should support at least this combination of flags.

user_26039090

Shared resources require protection from concurrent access because if multiple threads of execution access and manipulate the data at the same time, the threads may overwrite each other's changes or access data while it is in an inconsistent state. Concurrent access of shared data is a recipe for instability that often proves very hard to track down and debug—getting it right off the bat is important [1]

Threads can use the following two tools to synchronize their actions: mutexes and condition variables [2]

Mutexes (short for mutual exclusion) allow threads to synchronize their use of a shared resource, so that, for example, one thread doesn’t try to access a shared variable at the same time as another thread is modifying it.

Condition variables perform a complementary task: they allow threads to inform each other that a shared variable (or other shared resource) has changed state.

Adapted from:

[1] Love, R. (2005). Linux Kernel Development, Second Edition.

[2] Kerrish, M. (2010). The Linux Programming Interface.

user_56753250
Related