C++ share atomic_int64_t between different process?

user_9471941 0
Viewed 100

I am using C++ multi-processing passing data from one to another using shared memory. I put an array in the shared memory. Process A will copy data into the array, and Process B will use the data in the array. However, process B need to know how many item are in the array.

Currently I am using pipe/message queue to pass the array size from A to B. But I thinks I might put an atomic variable(like atomic_uint64_t) in the shared memory, modify it in process A and load it in process B. However I have the following questions.

  1. Is Atomic variable in C++ still atomic between process? I know atomic is implemented locking the cache line, neither another thread nor process can modify the atomic variable. So I think the answer is Yes.
  2. How exactly should I shared a atomic variable between? Can any one give an example?
atomic multiprocessing c++
2 Answers

Atomics will work, for the most part. Some caveats:

Technically, the behavior is non-portable (and lock-free does not guarantee address-free) but basic use including acquire-release should work on all mainstream platforms.

If you already have shared memory, the use should be pretty simple. Maybe something like this:

struct SharedBuffer
{
    std::atomic<std::size_t> filled;
    char buf[];
};

SharedBuffer* shared = static_cast<SharedBuffer*>(
      mmap(..., sizeof(SharedBuffer) + size, ...));

fill(shared->buf);
shared->filled.store(size, std::memory_order_release);

Note that you still have to solve the issue of notifying the other process. To the best of my knowledge, you cannot use std::condition variables and std::mutex. But the OS-specific types may work. For example for pthreads, you need to set pthread_mutexattr_setpshared and pthread_condattr_setpshared.

Maximizing portability

int64_t may be a bit risky if your CPU architecture is 32 bit and doesn't come with 64 bit atomics. You can check at runtime with atomic_is_lock_free or at compile time with is_always_lock_free.

Similarly, size_t may be risky if you want to mix 32 bit and 64 bit binaries. Then again, when targeting mixed binaries, you have to limit yourself to less than 32 bit address space anyway.

If you want to provide a fallback for missing lock-free atomics, atomic_flag is guaranteed to be lock-free. So you could roll your own spinlock. Personally, I wouldn't invest the time, however. You already use OS-facilities to set up shared memory. It is reasonable to make some assumptions about the runtime platform.

user_171673120

Thanks for the answers above. I write a simple code to verify whether we can simply use store(), load() and ++ in shared memory. The answer is YES. This is the code:

#include <string.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <atomic>

void* create_shared_memory(size_t size) {
    int protection = PROT_READ | PROT_WRITE;

    int visibility = MAP_SHARED | MAP_ANONYMOUS;

    return mmap(nullptr, size, protection, visibility, -1, 0);
}

struct SharedBuffer
{
    std::atomic<uint64_t> filled;
    int arr[1024];
};


int main() {
    void* shmem = create_shared_memory(sizeof(SharedBuffer));
    auto data = static_cast<SharedBuffer*>(shmem);
    data->filled.store(0);

    int pid = fork();

    uint64_t countInThisThread = 0;
    if (pid == 0) {
        while(data->filled.load() < 1024ULL * 1024) {
            if (data->filled.load() % 2 == 0) {
                data->filled++;
                countInThisThread++;
            }
        }
        printf("++ in child process: %lu\n", countInThisThread);
    } else {
        while(data->filled.load() < 1024ULL * 1024) {
            if (data->filled.load() % 2 == 1) {
                data->filled++;
                countInThisThread++;
            }
        }
        printf("++ in parent process: %lu\n", countInThisThread);
    }
    munmap(shmem, sizeof(SharedBuffer));
    return 0;
}
// Output
// ++ in parent process: 524288
// ++ in child process: 524288

I put a atomic<uint64_t> in the shared memory and add 1 to it in different process. Each process added 524288 times.

Be aware that this is only tested on 64 bit ubuntu 20.04 with g++7.5.0. More functions of atomic like wait/notify is not tested.

user_94719410
Related