CGO: converting Go byte array into C char* and back, issue with null terminator in byte array

user_13116539 0
Viewed 951

Brief Context: I am trying to create a cgo cache from uint64->[]byte for my company. Golang's map[uint64][]byte incur considerable latency due to the garbage collector as []byte is considered a pointer. As such, I would like to try to use CGO to avoid the issue. I am currently implementing a C++ unordered_map<unsigned long long, char*> to handle that. I managed to get the C++ wrappers to work but I am facing a major issue.

Currently, I am converting my Go byte array using

b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))

However, it turns out my my bb is []byte{1}. The 0 in the byte array is seen as the '/0' and thus shorten the string. Furthermore, it seems to have cause out of memory issue when I delete entries with

delete (map->find(key))->second. I suspect this is because that chars after the first '/0' does not get deallocated. I am not sure how else to do this. Personally, I am new to CGO so I never used it prior to this project so any help would be appreciated.

cgo go c-strings memory
1 Answers

There are two problems:

  • Use C.CBytes, not C.CString.

    Please read this in its entirety before using cgo.

  • The C.C* functions which allocate (C.CString and C.CBytes do that) internally call malloc() from the linked in libc library to get the destination memory block, and you're supposed to eventually call C.Free() on them (which calls libc's free()), as documented.

    AFAIK, C++ compiler is by no means oblidged to use libc's malloc() and free() to implement new and delete, so calling delete on the results of C.C* functions is a sure path to disaster.

    The simplest way to solve this, IMO, is to export a "constructor" function from your C++ side: something like

    extern "C" {
    char* clone(char *src, size_t len);
}

    …which would 1) allocate a memory block of length len using whatever method works best for C++; 2) copy len bytes from src to it; 3) return it.

    You could then call it from the Go side—as C.clone(&b[0], len(b)) and call it a day: the C++ side is free to call delete on the result.

user_7209990
Related