Where is the downside if the mutex and the data items are close together in memory?

Viewed 405

In 《c++ concurrency in action》page 265,

enter image description here

The data in the same cache-line with mutex-locks will be propagated because the mutex-locks are implemented as a r-m-w atomic operation. So the downside of this situation is the cost of ‘call to the operating system kernel’?

2 Answers

This is pretty low level computer architecture question so if you don't get something just ask questions. Knowing that let's get to the point.

Firstly, data in processor L1 caches is organized in so called cache lines. These are usually about 64 bytes long. Note that a single cache line is something indivisible for the cache - by which I mean it's 'marked' as valid or invalid as a whole.

Secondly modern processors have multiple cores and L1 cache is per-core cache. Therefore they need a way to synchronize their contents somehow. One of possible ways is MESI protocol and its 'children'. I suggest you read about it as it may help you understand my answer better. But back to the point - the MESI protocol may invalidate cache lines in one core (or even multiple cores) if other core modifies cache line refering to the same memory address. So far so good right?

So knowing these facts should make the answer pretty obvious now. If some not shared data (mutex in your example) lies near some shared data in memory it may get into the same cache line. This may cause unneeded and undesired performance hits for using this not shared memory block

bartop already has an important part of the answer: cache lines are typically managed using the MESI protocol. But it misses how MESI causes that effect.

MESI states are:

  • Modified (M)
  • Exclusive (E)
  • Shared (S)
  • Invalid (I)

The cache line containing the mutex must be Shared for other threads/cores to check the mutex state. The protected data however would benefit from being in Modified or Exclusive state in the core where the mutex is locked. As noted by bartop, the MESI state applies to the whole of a cache line. If the mutex and the protected data would share the same cache line, this could cause a large number of state transitions.

Related