Does atomic read guarantees reading of the latest value?

if we declare atomic variable, do we always get the latest value of the variable calling load() operation?

Yes, for some definition of latest.

The problem with concurrency is that it is not possible to argue about order of events in the usual way. This comes from a fundamental limitation in the hardware where the only way to establish a global order of operations across multiple cores would be to serialize them (and eliminating all of the performance benefits of parallel computation in the process).

What modern processors provide instead is an opt-in mechanism to re-establish order between certain operations. Atomics are the language-level abstraction for that mechanism. Imagine a scenario in which two atomic<int>s a and b are shared between threads (and let's further assume they were initialized to 0):

// thread #1
a.store(1);
b.store(1);

// thread #2
while(b.load() == 0) { /* spin */ }
assert(a.load() == 1);

The assertion here is guaranteed to hold. Thread #2 will observe the "latest" value of a.

What the standard does not talk about is when exactly the loop will observe the value of b changing from 0 to 1. We know it will happen some time after the write by thread #1 and we also know it will happen after the write to a. But we don't know how long after.

This kind of reasoning is further complicated by the fact, that different threads are allowed to disagree when certain writes took place. If you switch to a weaker memory ordering, one thread may observe writes to distinct atomic variables happening in a different order than what is observed by another thread.

When we talk about memory access on modern architectures, we usually ignore the "exact location" the value is read from.

A read operation can fetch data from the cache (L0/L1/...), the RAM or even the hard-drive (e.g. when the memory is swapped).

These keywords tell the compiler which assembly operations to use when accessing the data.

volatile

A keyword that tells the compiler to always read the variable's value from memory, and never from the register.

This "memory" can still be the cache, but, in case that this "address" in the cache is considered "dirty", meaning that the value has changed by a different processor, the value will be reloaded.

This ensures we never read a stale value.

Clarification: According to the standard, if the volatile type is not a primitive, whose read/write operations are atomic (in regard to the assembly instructions that read/write it) by nature, one might possibly read an intermediate value (the writer managed to write only half of the bytes by the time the reader read it). However, modern implementations do not behave this way.

atomic

When the compiler sees a load (read) operation, it basically does the exact same thing it would have done for a volatile value.

So, what is the difference???

The difference is cross-CPU write operations. When working with a volatile variable, if CPU 1 sets the value, and CPU 2 reads it, the reader might read an old value.

But, how can that be? The volatile keyword promises that we won't read a stale value!

Well, that's because the writer didn't publish the value! And though the reader tries to read it, it reads the old one.

When the compiler stumbles upon a store (write) operation for an atomic variable it:

• Sets the value atomically in memory
• Announces that the value has changed

After the announcement, all the CPUs will know that they should re-read the value of the variable because their caches will be marked "dirty".

This mechanism is very similar to operations performed on files. When your application writes to a file on the hard-drive, other applications may or may not see the new information, depending on whether or not your application flushed the data to the hard-drive.

If the data wasn't flushed, then it merely resides somewhere in your application's caches and visible only itself. Once you flush it, anyone who opens the file will see the new state.

Clarification: Common modern compiler & cache implementations ensure correct publishing of volatile writes as well. However, this is NOT a reason to prefer that over std::atomic. For example, just like some comments pointed out, Linux's atomic read and writes for x86_64 are implemented using volatiles.

As ComicSansMS's answer says, "latest" requires some definition of simultanaeity. Understanding how hardware cache coherence works can give you a better idea of what you're going to get in practice and why the C++ standard doesn't technically guarantee the "latest value" for operations that aren't serialized. Atomic RMW operations on the same atomic variable are necessarily serialized, hence there is a "latest value" guarantee for those, but that doesn't make it better if you just need to read.

e.g. maybe 40 nanoseconds for a store in one core to invalidate (MESI) the cache line before it can commit its store, so no other cores have a cached value they can read. (Of course they could have loaded at some earlier time before the invalidate, with out-of-order exec, but that's a small time window and blocking it would hurt the common fast case a lot.)

There's also a C++ guarantee that a consistent modification order exists for each atomic variable separately. And if you've seen one value for that variable, later reads in the same thread are guaranteed to see that value or later. (Read Read coherence and so on, 6.9.2.2 : 19 intro.races in the standard.)

A load will see a very recent value if there are ongoing stores

If there was only one recent store, it will see it or not

On real systems, if it was longer ago than maybe 100 nanoseconds, or maybe a microsecond or two in really high contention cases, loads in other threads will see it. (Where the time of the store is what an rdtsc would have seen if you'd done one in the same thread as the store. i.e. before it even retires and sends out a request to other cores to invalidate their copies.)

i.e. I'm proposing a definition of simultanaeity where the writer and reader both run an rdtsc instruction within a few cycles of when their store and load executes in the out-of-order back end. That's very different from when readers can actually expect to see stores from other threads.

Even a seq_cst atomic RMW doesn't wait for other cores to drain their store buffers (or make it happen any faster) to make executed but not committed stores visible, so it's not fundamentally better.

Re: "latest value" concerns, see the following.

• Is a memory barrier required to read a value that is atomically modified? (no)
• MESI Protocol & std::atomic - Does it ensure all writes are immediately visible to other threads?
• Does hardware memory barrier make visibility of atomic operations faster in addition to providing necessary guarantees?.

Another answer on this question suggests that stale data would be possible if the compilers didn't emit extra asm to explicitly "publish" stored data (make it globally visible). But all real systems have coherent cache across all the cores that C++ std::thread will start threads across. It's hypothetically possible to have std::thread run across cores with non-coherent shared memory, but would be extremely slow. See When to use volatile with multi threading? - never, obsoleted by C++11, but legacy code (and the Linux kernel) still use volatile to roll their own atomics.

Just a plain store instruction in assembly creates inter-core visibility because hardware is cache-coherent, using MESI. That's what you get from volatile. No "publish" is necessary. If you want this core to wait until the store is globally visible before doing later loads/stores, that's what a memory barrier does, to create ordering between this store and operations on other objects. Nothing to do with guaranteeing or speeding up visibility of this store.

The default std::memory_order is seq_cst; plain volatile is like relaxed on C++ implementations where it works for hand-rolled atomics. In ISO C++ volatile has undefined behaviour on data races, only atomic makes that safe. But real implementations, other than clang -fsanitize=thread or similar, don't do race detection.

Of course don't actually use volatile for threading. I mention this only to help understanding of how CPUs work, for thinking about performance and to help debugging accidental data races. C/C++11 made volatile obsolete for that purpose. Unless you're writing Linux kernel code (and then use their macros which just happen to use volatile under the hood).