How to return an error string from go to python

user_774909 0
Viewed 1001

I'm writing a shared object in Go (c-shared) which will be loaded and run from python. Everything is working fine, until the Go code needs to return an error. I am converting the error to string using error.Error() but when trying to return that to python, cgo is hitting:

panic: runtime error: cgo result has Go pointer

Which is very odd, since this is a string and not a pointer supposedly. I know there are no issues with returning go strings via shared object exported function, as I do that in several other places without any issue.

The Go code looks like:

package main

import "C"

//export MyFunction
func MyFunction() string {
    err := CallSomethingInGo()
    if err != nil {
        return err.Error()
    }
    return ""
}

func main() {}

The go code is compiled to .so using buildmode=c-shared and then In the python code, I have something like this:

from ctypes import *

lib = cdll.LoadLibrary("./mygocode.so")

class GoString(Structure):
    _fields_ = [("p", c_char_p),("n", c_longlong)]

theFunction = lib.MyFunction
theFunction.restype = GoString

err = theFunction()

When the last line executes and the golang code returns NO error then everything is fine and it works! But, if the golang code tries to return an error (e.g. CallSomethingInGo fails and returns err) then the python code fails with:

panic: runtime error: cgo result has Go pointer

I've tried manually returning strings from go to python and it works fine, but trying to return error.Error() (which should be a string per my understanding) fails. What is the correct way to return the string representation of the error to python?

One more piece of info - from golang, I did a printf("%T", err) and I see the type of the error is:

*os.PathError

I also did printf("%T", err.Error()) and confirmed the type returned by err.Error() was 'string' so I am still not sure why this isn't working.

Even stranger to me...I tried modifying the go functions as shown below for a test, and this code works fine and returns "test" as a string back to python...

//export MyFunction
func MyFunction() string {
    err := CallSomethingInGo()
    if err != nil {
        // test
        x := errors.New("test")
        return x.Error()
    }
    return ""
}

I'm so confused! How can that test work, but not err.Error() ?

cgo go error-handling python string
1 Answers

As I said in a comment, you're just not allowed to do that.

The rules for calling Go code from C code are outlined in the Cgo documentation, with this particular issue described in this section, in this way (though I have bolded a few sections in particular):

Passing pointers

Go is a garbage collected language, and the garbage collector needs to know the location of every pointer to Go memory. Because of this, there are restrictions on passing pointers between Go and C.

In this section the term Go pointer means a pointer to memory allocated by Go (such as by using the & operator or calling the predefined new function) and the term C pointer means a pointer to memory allocated by C (such as by a call to C.malloc). Whether a pointer is a Go pointer or a C pointer is a dynamic property determined by how the memory was allocated; it has nothing to do with the type of the pointer.

Note that values of some Go types, other than the type's zero value, always include Go pointers. This is true of string, slice, interface, channel, map, and function types. A pointer type may hold a Go pointer or a C pointer. Array and struct types may or may not include Go pointers, depending on the element types. All the discussion below about Go pointers applies not just to pointer types, but also to other types that include Go pointers.

Go code may pass a Go pointer to C provided the Go memory to which it points does not contain any Go pointers. The C code must preserve this property: it must not store any Go pointers in Go memory, even temporarily. When passing a pointer to a field in a struct, the Go memory in question is the memory occupied by the field, not the entire struct. When passing a pointer to an element in an array or slice, the Go memory in question is the entire array or the entire backing array of the slice.

C code may not keep a copy of a Go pointer after the call returns. This includes the _GoString_ type, which, as noted above, includes a Go pointer; _GoString_ values may not be retained by C code.

A Go function called by C code may not return a Go pointer (which implies that it may not return a string, slice, channel, and so forth). A Go function called by C code may take C pointers as arguments, and it may store non-pointer or C pointer data through those pointers, but it may not store a Go pointer in memory pointed to by a C pointer. A Go function called by C code may take a Go pointer as an argument, but it must preserve the property that the Go memory to which it points does not contain any Go pointers.

Go code may not store a Go pointer in C memory. C code may store Go pointers in C memory, subject to the rule above: it must stop storing the Go pointer when the C function returns.

These rules are checked dynamically at runtime. The checking is controlled by the cgocheck setting of the GODEBUG environment variable. The default setting is GODEBUG=cgocheck=1, which implements reasonably cheap dynamic checks. These checks may be disabled entirely using GODEBUG=cgocheck=0. Complete checking of pointer handling, at some cost in run time, is available via GODEBUG=cgocheck=2.

It is possible to defeat this enforcement by using the unsafe package, and of course there is nothing stopping the C code from doing anything it likes. However, programs that break these rules are likely to fail in unexpected and unpredictable ways.

This is what you are seeing: you have a program that breaks several rules, and now it fails in unexpected and unpredictable ways. In particular, your lib.MyFunction is

a Go function called by C code

since Python's cdll handlers count as C code. You can return nil, as that's the zero-value, but you are not allowed to return Go strings. The fact that the empty-string constant (and other string constants from some other error types) is not caught at runtime is a matter of luck.1

1Whether this is good luck or bad luck depends on your point of view. If it failed consistently, perhaps you would have consulted the Cgo documentation earlier. Instead, it fails unpredictably, but not in your most common case. What's happening here is that the string constants were compiled to text (or rodata) sections and therefore are not actually dynamically allocated. However, some—not all, but some—errors' string bytes are dynamically allocated. Some os.PathErrors point into GC-able memory, and these are the cases that are caught by the

reasonably cheap dynamic checks

mentioned in the second-to-last paragraph.

user_12564520
Related