Proper way to write to C memory from go

Question

I would like to port a go program as a library by exporting some C APIs. In C it's common for a function to accept a buffer (pointer) as parameter (along with its size) and write the output to it. For example (from C standard library)

size_t fread(void *ptr, size_t size, size_t nmemb, FILE *stream);

This will write to the *ptr specified by the caller for at most size*nmemb bytes. The benefit of such design is that memory is managed by the caller, and thus the function does not need to deal with memory allocation. I want my API exported from go to be look like this as well.

However, I found there is no direct way to write to a pointer in go. So I created something like this (slight modification of case 6 of this documentation) that creates a []byte object in go using the underlying memory of C.

// create a byte array using C memory for internal use
func createBuffer(buf unsafe.Pointer, size int) []byte {
    var res []byte
    hdr := (*reflect.SliceHeader)(unsafe.Pointer(&res))
    hdr.Data = uintptr(unsafe.Pointer(buf))
    hdr.Len = size
    hdr.Cap = size
    return res
}

I then exported my function as the following (readers is a slice of io.Reader, unsafe.Pointer will have type void* after export) that will read some data to the C memory provided with at most size bytes.

//export read
func read(id int, buf unsafe.Pointer, size C.int) int {
    res, err := readers[id].Read(createBuffer(buf, int(size)))  // works
    //res, err := readers[id].Read(C.GoBytes(buf, size)) // does not work
    if err != nil {
        return -1
    }
    return res
}

This works as expected (i.e. in my C program, the buffer does contain the desired data after invoking read), but I am not sure whether this is the correct way to do so (i.e. whether there are memory issues especially related to garbage collection (GC)).

Suppose the resulted []byte returned from createBuffer is not used anywhere else after read terminated, how will the memory get deallocated by GC? Will the GC also attempting to free hdr.Data, which is supposed to be a piece of memory managed by the C program? The official documentation provides quite a bit of warning about this. If this will cause GC problems, what is the correct way to do so? (I already tried C.GoBytes(...), it seems to copy the underlying buffer and thus does not work)

Proper way to write to C memory from go

0 Answers