How to pass a Java string array []String to Go via cgo/JNA

user_486158 0
Viewed 224

I would like to pass a String[] in Java to my Go function via JNA.

My go function has the following signature:

func PredicateEval(keys, values []string, expression string) *C.char

I have compiled the go library with link mode as "c-shared". I have a GoString in java defined as:

package predicates;
import com.ochafik.lang.jnaerator.runtime.NativeSize;
import com.sun.jna.Pointer;
import com.sun.jna.Structure;
import java.util.Arrays;
import java.util.List;
/**
 * <i>native declaration : coverage_server/predicate_jvm_bridge/lib/libtest.h</i><br>
 * This file was autogenerated by <a href="http://jnaerator.googlecode.com/">JNAerator</a>,<br>
 * a tool written by <a href="http://ochafik.com/">Olivier Chafik</a> that <a href="http://code.google.com/p/jnaerator/wiki/CreditsAndLicense">uses a few opensource projects.</a>.<br>
 * For help, please visit <a href="http://nativelibs4java.googlecode.com/">NativeLibs4Java</a> , <a href="http://rococoa.dev.java.net/">Rococoa</a>, or <a href="http://jna.dev.java.net/">JNA</a>.
 */
public class _GoString_ extends Structure {
    /** C type : const char* */
    public Pointer p;
    public NativeSize n;
    public _GoString_() {
        super();
    }
    protected List<String> getFieldOrder() {
        return Arrays.asList("p", "n");
    }
    /** @param p C type : const char* */
    public _GoString_(Pointer p, NativeSize n) {
        super();
        this.p = p;
        this.n = n;
    }
    public static class ByReference extends _GoString_ implements Structure.ByReference {
        
    };
    public static class ByValue extends _GoString_ implements Structure.ByValue {
        
    };
}

I also have a Go slice defined as:

package predicates;
import com.sun.jna.Pointer;
import com.sun.jna.Structure;
import java.util.Arrays;
import java.util.List;
/**
 * <i>native declaration : coverage_server/predicate_jvm_bridge/lib/libtest.h</i><br>
 * This file was autogenerated by <a href="http://jnaerator.googlecode.com/">JNAerator</a>,<br>
 * a tool written by <a href="http://ochafik.com/">Olivier Chafik</a> that <a href="http://code.google.com/p/jnaerator/wiki/CreditsAndLicense">uses a few opensource projects.</a>.<br>
 * For help, please visit <a href="http://nativelibs4java.googlecode.com/">NativeLibs4Java</a> , <a href="http://rococoa.dev.java.net/">Rococoa</a>, or <a href="http://jna.dev.java.net/">JNA</a>.
 */
public class GoSlice extends Structure {
    /** C type : void* */
    public Pointer data;
    /** C type : GoInt */
    public long len;
    /** C type : GoInt */
    public long cap;
    public GoSlice() {
        super();
    }
    protected List<String> getFieldOrder() {
        return Arrays.asList("data", "len", "cap");
    }
    /**
     * @param data C type : void*<br>
     * @param len C type : GoInt<br>
     * @param cap C type : GoInt
     */
    public GoSlice(Pointer data, long len, long cap) {
        super();
        this.data = data;
        this.len = len;
        this.cap = cap;
    }
    public static class ByReference extends GoSlice implements Structure.ByReference {
        
    };
    public static class ByValue extends GoSlice implements Structure.ByValue {
        
    };
}

This is my attempt to convert a Java []String to a Go string[].

  static {
    try {
      Field field = sun.misc.Unsafe.class.getDeclaredField("theUnsafe");
      field.setAccessible(true);
      unsafe = (sun.misc.Unsafe) field.get(null);
      Class<?> clazz = ByteBuffer.allocateDirect(0).getClass();
      DIRECT_BYTE_BUFFER_ADDRESS_OFFSET = unsafe.objectFieldOffset(Buffer.class.getDeclaredField("address"));
      DIRECT_BYTE_BUFFER_CLASS = clazz;
    } catch (Exception e) {
      throw new AssertionError(e);
    }
  }

  private static long getAddress(ByteBuffer buffer) {
    assert buffer.getClass() == DIRECT_BYTE_BUFFER_CLASS;
    return unsafe.getLong(buffer, DIRECT_BYTE_BUFFER_ADDRESS_OFFSET);
  }

  public static _GoString_.ByValue JavaStringToGo(String jstr) {
    try {
      byte[] bytes = jstr.getBytes("utf-8");
      //ByteBuffer bb = ByteBuffer.wrap(bytes);
      ByteBuffer bb = ByteBuffer.allocateDirect(bytes.length);
      bb.put(bytes);
      Pointer p = new Pointer(getAddress(bb));
      _GoString_.ByValue value = new _GoString_.ByValue();

      value.n = new NativeSize(bytes.length);
      value.p = p;
      return value;
    } catch (UnsupportedEncodingException e) {
      throw new RuntimeException(e);
    }
  }
  
  public static GoSlice.ByValue JavaStringArrayToGoStringSlice(String[] strings) {
    _GoString_.ByValue[] goStrings = new _GoString_.ByValue[strings.length];
    for (int i = 0; i < strings.length; i++) {
      goStrings[i] = JavaStringToGo(strings[i]);
    }

    Memory arr = new Memory(strings.length * Native.getNativeSize(_GoString_.class));
    for (int i = 0; i < strings.length; i++) {
      System.out.println(Native.getNativeSize(_GoString_.class));
      byte[] bytes = goStrings[0].getPointer().getByteArray(0, Native.getNativeSize(_GoString_.class));
      arr.write(i*Native.getNativeSize(_GoString_.class), bytes, 0, bytes.length);
    }
    GoSlice.ByValue slice = new GoSlice.ByValue();
    slice.data = arr;
    slice.len = strings.length;
    slice.cap = strings.length;

    return slice;
  }

Everything compiles, but when i try to access the slice elements on the Go side, i get a seg fault:

unexpected fault address 0xb01dfacedebac1e
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x1 addr=0xb01dfacedebac1e pc=0x10d7d3d6f]

goroutine 17 [running, locked to thread]:
jna cgo go c java
1 Answers

You're losing track (and control) of the memory allocation of the actual Strings.

Your mapping of _GoString_ only includes allocation of a Pointer (4 or 8 bytes) and a NativeSize (a 4- or 8-byte size_t). This mapping assumes the Pointer remains valid:

public class _GoString_ extends Structure {
    /** C type : const char* */
    public Pointer p;
    public NativeSize n;
    // constructors, etc.
}

However, when you assign the value to p you only keep track of the pointer's address and not the actual memory allocation (I've added comments to your code):

  public static _GoString_.ByValue JavaStringToGo(String jstr) {
    try {
      byte[] bytes = jstr.getBytes("utf-8");
      // 
      // Here you allocate memory for the bytes
      //
      ByteBuffer bb = ByteBuffer.allocateDirect(bytes.length);
      bb.put(bytes);
      // 
      // Here you only keep track of the pointer to the bytes
      //
      Pointer p = new Pointer(getAddress(bb));
      // 
      // You never reference bb again, it is no longer reachable
      // and its allocation can be reclaimed by the system
      //
      _GoString_.ByValue value = new _GoString_.ByValue();

      value.n = new NativeSize(bytes.length);
      value.p = p;
      return value;
    } catch (UnsupportedEncodingException e) {
      throw new RuntimeException(e);
    }
  }

As a direct bytebuffer, the memory location and (de)allocation mechanism are different than normal objects and GC, but the fundamental principle applies that once you've lost reference to the Java object (the ByteBuffer) you have no control over when that native memory will be freed. (When bb is GC'd, its internal fields include the reference that will trigger the deallocation when it is processed.)

One possible solution is to add a private field to your _GoString_ class that maintains a strong reference to the ByteBuffer and prevents the system from reclaiming its memory (possibly adding a ByteBuffer constrcutor).

Another solution is to use JNA's Memory class for the Strings and directly store that Memory object (which extends Pointer) to the p field. I'm not sure why you have chosen direct byte buffers for this application, so this may not apply to your use case but it would certainly simplify your code.

user_11614840
Related