Shared lock with two exclusive lock groups

Based on the reply from alexb I have written the following mutex class which currently works for me (only tried out in a simple multithreaded example application so far)

Please note that it is not protected against "starvation". In simple words: It is not ensured that that lockMeasure will ever get the lock if lockLogging is called high-frequently (and the other way round).

class MyMutex
{
private:
    std::atomic<int> log_executors;
    std::atomic<int> measure_executors;

    std::mutex mtx;
    std::condition_variable condition;

public:
    MyMutex() : log_executors(0), measure_executors(0) {}
    
    ~MyMutex() {}

    void lockMeasure()
    {   
        std::unique_lock<std::mutex> lock(mtx);

        while(log_executors) {
            condition.wait(lock); 
        }
        measure_executors++; 
    }
    
    void unlockMeasure()
    {   
        std::unique_lock<std::mutex> lock(mtx);

        measure_executors--; 
        if (!measure_executors)
        {
          condition.notify_all();
        }
    }
    
    void lockLogging()
    {         
        std::unique_lock<std::mutex> lock(mtx);

        while(measure_executors) {
          condition.wait(lock); 
        }
        log_executors++;
    }

    void unlockLogging()
    {         
        std::unique_lock<std::mutex> lock(mtx);

        log_executors--; 
        if (!log_executors)
        {
          condition.notify_all(); 
        }
    }

    static MyMutex& getInstance()
    {
        static MyMutex _instance;
        return _instance;
    }    
};

Usage:

void measure()
{
    MyMutex::getInstance().lockMeasure();

    someMeasureFunctionality();

    MyMutex::getInstance().unlockMeasure();
}

void log()
{
    MyMutex::getInstance().lockLogging();

    someLogFunctionality();

    MyMutex::getInstance().unlockLogging();
}

You need some barrier logic which is more complicated than shared_mutex (BTW, shared_mutex is not best choice for multiplatform compilation). For example, you can use mutex, conditional variable, and 2 variables for barrier sync. It does not take CPU and you may not use sleeps for check.

#include <mutex>
#include <condition_variable>
#include <atomic>

std::atomic<int> log_executors = 0;
std::atomic<int> measure_executors = 0;

std::mutex mutex;
std::condition_variable condition;

void log(std::string message) {
  {
    std::unique_lock<std::mutex> lock(mutex);

    log_executors++;  // Register current executor and prevent from entering new measure executors

    // Wait until all measure executors will go away
    while(measure_executors) {
      condition.wait(lock);  // wait condition variable signal. Mutex will be unlocked during wait
    }
  }

  // here lock is freed
  someLogFunctionality(); // execute logic


  {
    std::unique_lock<std::mutex> lock(mutex);
    log_executors--;  // unregister current execution
    condition.notify_all();  // send signal and unlock all waiters
  }
}

void measure()
{        
  {
    std::unique_lock<std::mutex> lock(mutex);

    measure_executors++;  // Register current executor and prevent from entering new log executors
    while(log_executors) {
      condition.wait(lock);  // wait until all measure executors will gone
    }
  }

  someMeasureFunctionality();

  {
    std::unique_lock<std::mutex> lock(mutex);
    measure_executors--;  // unregister current execution
    condition.notify_all(); // send signal and unlock all waiters
  }
}

You can have a master lock granting access to a semaphore variable:

void log(std::string message)
{
    acquire(LOG);
    someLogFunctionality();
    release(LOG);
}

void measure()
{        
    acquire(MEASURE);
    someMeasureFunctionality();
    release(MEASURE);
}

void acquire(int what) {
    for (;;) {
        mtx.lock();
        if (owner == NONE) {
            owner = what;
        }
        if (owner == what) {
            // A LOG was asked while LOG is running
            users[owner]++;
            mtx.unlock();
            return;
        }
        mtx.unlock();
        // Some sleep would be good
        usleep(5000);
    }
}

void release(int what) {
    mtx.lock();
    if (owner != what) {
        // This is an error. How could this happen?
    }
    if (users[what] <= 0) {
        // This is an error. How could this happen?
    }
    users[what]--;
    if (0 == users[what]) {
        owner = NONE;
    }
    mtx.unlock();
}

In this case, for example:

owner is NONE
LOG1 acquires LOG. It can do so because owner is NONE
MEASURE1 acquires LOG. It starts spinning in place because owner != MEASURE
MEASURE2 acquires LOG. It starts spinning in place because owner != MEASURE
LOG2 acquires LOG. It can do so because owner is LOG, users[LOG]=2
LOG2 releases LOG. users[LOG]=1
LOG1 releases LOG. users[LOG]=0, so owner becomes NONE
MEASURE2 by pure chance acquires mtx before MEASURE1, finds owner=NONE and goes
MEASURE1 finds owner=MEASURE and sets users[MEASURE]=2

In the above, note that the second call to measure() actually executed a bit earlier. This should be OK. But if you want to keep the calls "serialized" even if they happen in parallel, you'll need a stack for each owner and more complex code.