Valgrind and CUDA: Are reported leaks real?

user_694195 0
Viewed 8436

I have a very simple CUDA component in my application. Valgrind reports a lot of leaks and still-reachables, all related to the cudaMalloc calls.

Are these leaks real? I call cudaFree for every cudaMalloc. Is this valgrind's inability to interpret GPU memory allocation? If these leaks are not real, can I suppress them and have valgrind only analyse the non-gpu part of the application?

extern "C"
unsigned int *gethash(int nodec, char *h_nodev, int len) {
    unsigned int *h_out = (unsigned int *)malloc(sizeof(unsigned int) * nodec);

    char *d_in;
    unsigned int *d_out;

    cudaMalloc((void**) &d_in, sizeof(char) * len * nodec);
    cudaMalloc((void**) &d_out, sizeof(unsigned int) * nodec);

    cudaMemcpy(d_in, h_nodev, sizeof(char) * len * nodec, cudaMemcpyHostToDevice);

    int blocks = 1 + nodec / 512;


    cube<<<blocks, 512>>>(d_out, d_in, nodec, len);

    cudaMemcpy(h_out, d_out, sizeof(unsigned int) * nodec, cudaMemcpyDeviceToHost);

    cudaFree(d_in);
    cudaFree(d_out);
    return h_out;

}

Last bit of the Valgrind output:

...
==5727== 5,468 (5,020 direct, 448 indirect) bytes in 1 blocks are definitely lost in loss record 506 of 523
==5727==    at 0x402B965: calloc (in /usr/lib/valgrind/vgpreload_memcheck-x86-linux.so)
==5727==    by 0x4843910: ??? (in /usr/lib/nvidia-319-updates/libcuda.so.319.60)
==5727==    by 0x48403E9: ??? (in /usr/lib/nvidia-319-updates/libcuda.so.319.60)
==5727==    by 0x498B32D: ??? (in /usr/lib/nvidia-319-updates/libcuda.so.319.60)
==5727==    by 0x494A6E4: ??? (in /usr/lib/nvidia-319-updates/libcuda.so.319.60)
==5727==    by 0x4849534: ??? (in /usr/lib/nvidia-319-updates/libcuda.so.319.60)
==5727==    by 0x48191DD: cuInit (in /usr/lib/nvidia-319-updates/libcuda.so.319.60)
==5727==    by 0x406B4D6: ??? (in /usr/lib/i386-linux-gnu/libcudart.so.5.0.35)
==5727==    by 0x406B61F: ??? (in /usr/lib/i386-linux-gnu/libcudart.so.5.0.35)
==5727==    by 0x408695D: cudaMalloc (in /usr/lib/i386-linux-gnu/libcudart.so.5.0.35)
==5727==    by 0x804A006: gethash (hashkernel.cu:36)
==5727==    by 0x804905F: chkisomorphs (bdd.c:326)
==5727== 
==5727== LEAK SUMMARY:
==5727==    definitely lost: 10,240 bytes in 6 blocks
==5727==    indirectly lost: 1,505 bytes in 54 blocks
==5727==      possibly lost: 7,972 bytes in 104 blocks
==5727==    still reachable: 626,997 bytes in 1,201 blocks
==5727==         suppressed: 0 bytes in 0 blocks
cuda memory-leaks valgrind
5 Answers

Since I don't have 50 reputation, I cannot leave a comment on @Vyas 's answer.

I feel strange that cuda-memcheck cannot observe cuda memory leakage.

I just write a very simple code with a cuda memory leakage, but when using cuda-memcheck --leak-check full it give no leakage. It is:

#include <iostream>
#include <cuda_runtime.h>

using namespace std;

int main(){
    float* cpu_data;
    float* gpu_data;
    int buf_size = 10 * sizeof(float);

    cpu_data = (float*)malloc(buf_size);
    for(int i=0; i<10; i++){
        cpu_data[i] = 1.0f * i;
    }

    cudaError_t cudaStatus = cudaMalloc(&gpu_data, buf_size);

    cudaMemcpy(gpu_data, cpu_data, buf_size, cudaMemcpyHostToDevice);

    free(cpu_data);
    //cudaFree(gpu_data);

    return 0;
}

Note the commented line of code, which make this program a cuda memory leakage, I think. However, when execuing cuda-memcheck ./a.out it gives:

========= CUDA-MEMCHECK
========= ERROR SUMMARY: 0 errors
user_29990960
Related