I'm playing a bit with cudaDeviceSetLimit(), trying to see the maximum size allowed to set for my GPU.
Here I get unexpected results:
The Heap size stops increasing after 17.6 Gb, so I guess this is the maximum value I can set. But since my GPU has only 16 Gb of memory (as confirmed by querying the device props), how is it possible for the Heap size to be greater?
The limit is not related to the actual memory size. Just because you can set a particular limit does not mean that you will have that much available memory to allocate. This should be self-evident. I can set the heap limit to say 14GB, and then do a cudaMalloc operation for 14GB. If my GPU only has 16GB, both operations can succeed, but that does not mean I actually have 14GB of heap space "reserved".
The amount of heap space available is the lesser of the heap limit (minus any heap allocations already done) and the actual available GPU memory at that point.
