How to perform atomic operations on Linux that work on x86, arm, GCC and icc?

Viewed 29339

Every Modern OS provides today some atomic operations:

  • Windows has Interlocked* API
  • FreeBSD has <machine/atomic.h>
  • Solaris has <atomic.h>
  • Mac OS X has <libkern/OSAtomic.h>

Anything like that for Linux?

  • I need it to work on most Linux supported platforms including: x86, x86_64 and arm.
  • I need it to work on at least GCC and Intel Compiler.
  • I need not to use 3rd par library like glib or qt.
  • I need it to work in C++ (C not required)

Issues:

  • GCC atomic builtins __sync_* are not supported on all platforms (ARM) and are not supported by the Intel compiler.
  • AFAIK <asm/atomic.h> should not be used in user space and I haven't successfully used it at all. Also, I'm not sure if it would work with Intel compiler.

Any suggestions?

I know that there are many related questions but some of them point to __sync* which is not feasible for me (ARM) and some point to asm/atomic.h.

Maybe there is an inline assembly library that does this for GCC (ICC supports gcc assembly)?

Edit:

There is a very partial solution for add operations only (allows implementing atomic counter but not lock free-structures that require CAS):

If you use libstc++ (Intel Compiler uses libstdc++) then you can use __gnu_cxx::__exchange_and_add that defined in <ext/atomicity.h> or <bits/atomicity.h>. Depends on compiler version.

However I'd still like to see something that supports CAS.

9 Answers

Projects are using this:

http://packages.debian.org/source/sid/libatomic-ops

If you want simple operations such as CAS, can't you just just use the arch-specific implementations out of the kernel, and do arch checks in user-space with autotools/cmake? As far as licensing goes, although the kernel is GPL, I think it's arguable that the inline assembly for these operations is provided by Intel/AMD, not that the kernel has a license on them. They just happen to be in an easily accessible form in the kernel source.

Darn. I was going to suggest the GCC primitives, then you said they were off limits. :-)

In that case, I would do an #ifdef for each architecture/compiler combination you care about and code up the inline asm. And maybe check for __GNUC__ or some similar macro and use the GCC primitives if they are available, because it feels so much more right to use those. :-)

You are going to have a lot of duplication and it might be difficult to verify correctness, but this seems to be the way a lot of projects do this, and I've had good results with it.

Some gotchas that have bit me in the past: when using GCC, don't forget "asm volatile" and clobbers for "memory" and "cc", etc.

Boost, which has a non intrusive license, and other frameworks already offer portable atomic counters -- as long as they are supported on the target platform.

Third party libraries are good for us. And if for strange reasons your company forbid you from using them, you can still have a look at how they proceed (as long as the licence permit it for your use) to implement what your are looking for.

I recently did an implementation of such a thing and I was confronted to the same difficulties as you are. My solution was basically the following:

  • try to detect the gcc builtins with the feature macro
  • if not available just implement something like cmpxch with __asm__ for the other architectures (ARM is a bit more complicated than that). Just do that for one possible size, e.g sizeof(int).
  • implement all other functionality on top of that one or two primitives with inline functions

There is a patch for GCC here to support ARM atomic operations. WIll not help you on Intel, but you could examine the code - there is recent kernel support for older ARM architectures, and newer ones have the instructions built in, so you should be able to build something that works.

http://gcc.gnu.org/ml/gcc-patches/2011-07/msg00050.html

__sync* certainly is (and has been) supported by the Intel compiler, because GCC adopted these build-ins from there. Read the first paragraph on this page. Also see "IntelĀ® C++ Compiler for Linux* Intrinsics Reference", page 198. It's from 2006 and describes exactly those built-ins.

Regarding ARM support, for older ARM CPUs: it cannot be done entirely in userspace, but it can be done in kernelspace (by disabling interrupts during the operation), and I think I read somewhere that it is supported for quite a while now.

According to this PHP bug, dated 2011-10-08, __sync_* will only fail on

  • PA-RISC with anything other than Linux
  • SPARCv7 and lower
  • ARM with GCC < 4.3
  • ARMv5 and lower with anything other than Linux
  • MIPS1

So with GCC > 4.3 (and 4.7 is the current one), you shouldn't have a problem with ARMv6 and newer. You shouldn't have no problem with ARMv5 either as long as compiling for Linux.

Related