I was wondering if there is a complete list of atomic operations usable in CUDA kernels. I couldn't find something like that on the internet.
2 Answers
As of April 2020 (i.e. CUDA 10.2, Turing michroarchitecture), these are:
- addition
- subtraction
- minimum
- maximum
- bitwise-and
- bitwise-or
- bitwise-xor
- increment (with a wraparound value)
- decrement (with a wraparound value)
- compare-and-swap - which is perhaps the most significant, as you can "implement" essentially any atomic operation using compare-and-swap.
Note, however, that:
- Only certain data types are directly supported (of size never above 8 bytes).
- Earlier micro-architectures support less operations and/or less types.
- CUDA memory only supports aligned accesses - whether they be regular or atomic.
For details, consult the Atomic Functions section of the CUDA Programming guide.
Related