CUDA's handling of floating-point rounding modes is discussed here. While various intrinsics such as __fadd_rn are available for performing rounded operations within the normal flow of the program (adding with round-to-nearest here), the rounding mode of atomics such as atomicAdd don't seem to be specified and a method for setting this rounding mode isn't discussed.
Is it possible to set the rounding mode of CUDA atomics?
You can't.
Quoting from the PTX instruction set documentation:
The floating-point operation
.addoperation rounds to nearest even. Current implementation ofatom.add.f32on global memory flushes subnormal inputs and results to sign-preserving zero; whereas atom.add.f32 on shared memory supports subnormal inputs and results and doesn't flush them to zero.
atom.add.f16andatom.add.f16x2operation requires the.noftzqualifier; it preserves subnormal inputs and results, and does not flush them to zero.
Another way to think of it: The limited atomic floating support available in current CUDA hardware is done by an ALU in the memory controller, not by the floating point hardware in the SMs. The latter has all the rounding modes implemented, the former doesn't.
