How can I use JNA to pass byte[][] to go

Question

This is go code

import (
    "C"
    "fmt"
)
//export Print
func Print(keys, values [][]byte) {
  for len(keys) > 0 {
  err := txn.Set(keys[0], values[0])
  errMustBeNil(err)
  fmt.Printf("%s %s", string(keys[0]), string(values[0]))
  keys = keys[1:]
  values = values[1:]
 }
}

This is libPrint.h

/*
  static assertion to make sure the file is being used on architecture
  at least with matching size of GoInt.
*/
typedef char _check_for_64_bit_pointer_matching_GoInt[sizeof(void*)==64/8 ? 1:-1];

#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef _GoString_ GoString;
#endif
typedef void *GoMap;
typedef void *GoChan;
typedef struct { void *t; void *v; } GoInterface;
typedef struct { void *data; GoInt len; GoInt cap; } GoSlice;

#endif

/* End of boilerplate cgo prologue.  */

#ifdef __cplusplus
extern "C" {
#endif

extern void Print(GoSlice keys, GoSlice values);

#ifdef __cplusplus
}
#endif

This is my Java code,I want to put byte[] into pointer one by one,GoSlice is represents the array[] in go,I don't know if this is correct

public interface Test extends Library {
  Test TEST = (Test) Native.load("Print", Test.class);

  void Print(GoSlice key, GoSlice val);

  class GoSlice extends Structure {
    public Pointer[] data;
    public long len;
    public long cap;
  }
  static void main(String[] args) {
    byte[] byte1 = "key1".getBytes(StandardCharsets.UTF_8);
    byte[] byte2 = "value1new".getBytes(StandardCharsets.UTF_8);

    GoSlice keys = new GoSlice();
    keys.data = new Pointer[1];
    keys.data[0] = new Pointer(byte1.length + 1);
    keys.len = 1;
    keys.cap = 1;

    GoSlice values = new GoSlice();
    values.data = new Pointer[1];
    keys.data[0] = new Pointer(byte2.length + 1);
    values.len = 1;
    values.cap = 1;

    keys.data[0].write(0, byte1, 0, byte1.length);
    values.data[0].write(0, byte2, 0, byte2.length);

    Test.TEST.Print(keys, values);
  }
}

But it doesn't work correctly,I feel that byte[][] is not correctly converted to the type in go.

This is the console log after running the Java program

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00000001a2775a34, pid=64329, tid=0x0000000000002903
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.62.0.19-CA-macos-aarch64) (8.0_332-b09) (build 1.8.0_332-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.332-b09 mixed mode bsd-aarch64 compressed oops)
# Problematic frame:
# C  [libsystem_platform.dylib+0x3a34]  _platform_memmove+0xf4
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/user/project/hs_err_pid64329.log
#
# If you would like to submit a bug report, please visit:
#   http://www.azul.com/support/
#

This is my first change

public interface TxnClient extends Library {
  TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);

  int TxnSave(GoSlice[] key, GoSlice[] val);

  class GoSlice extends Structure {
    public long cap;
    public Pointer data;
    public long len;

    @Override
    protected List getFieldOrder() {
      return Arrays.asList("cap", "data", "len");
    }
  }

  static void main(String[] args) {
    byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
    byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
    byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
    byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);

    GoSlice tmp = new GoSlice();
    GoSlice[] keys = (GoSlice[]) tmp.toArray(2);
    keys[0].data = new Memory(key1.length + 1);
    keys[0].len = keys[0].cap = key1.length;
    keys[0].data.write(0, key1, 0, key1.length);

    keys[1].data = new Memory(key2.length + 1);
    keys[1].len = keys[1].cap = key2.length;
    keys[1].data.write(0, key2, 0, key2.length);

    GoSlice[] values = (GoSlice[]) tmp.toArray(2);
    values[0].data = new Memory(value1.length + 1);
    values[0].len = values[0].cap = value1.length;
    values[0].data.write(0, value1, 0, value1.length);

    values[1].data = new Memory(value2.length + 1);
    values[1].len = values[1].cap = value2.length;
    values[1].data.write(0, value2, 0, value2.length);

    int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
    System.out.println(i);
  }
}

5431788016 5431788016panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x9 pc=0x1225b2b80]

goroutine 17 [running, locked to thread]:
main.TxnSave({0x9, 0x143c281f0, 0x14000182000?}, {0x9, 0x143c281f0, 0x121daf2c8?})
    /Users/user/project/print.go:52 +0x180

second change

//export TxnSave
func TxnSave(keys, values [][]byte) (res int) {
 // ignore
}

public interface TxnClient extends Library {
  TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);

  int TxnSave(GoSlice key, GoSlice val);

  class GoSlice extends Structure {
    public long cap;
    public Pointer data;
    public long len;

    @Override
    protected List getFieldOrder() {
      return Arrays.asList("cap", "data", "len");
    }
  }

  static void main(String[] args) {
    byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
    byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
    byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
    byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);

    GoSlice keys = new GoSlice();
    keys.data = new Memory(key1.length + key2.length + 1);
    keys.data.write(0, key1, 0, key1.length);
    keys.len = keys.cap = key1.length + key2.length;
    keys.data.write(key1.length, key2, 0, key2.length);

    GoSlice values = new GoSlice();
    values.data = new Memory(value1.length + value2.length + 1);
    values.len = values.cap = value1.length + value2.length;
    values.data.write(0, value1, 0, value1.length);
    values.data.write(value1.length, value2, 0, value2.length);

    int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
    System.out.println(i);
  }
}

[2022/07/31 22:36:57.225 +08:00] [INFO] [client.go:378] ["[pd] create pd client with endpoints"] [pd-address="[127.0.0.1:2379]"]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:350] ["[pd] switch leader"] [new-leader=http://127.0.0.1:2379] [old-leader=]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:105] ["[pd] init cluster id"] [cluster-id=7126545341327379321]
[2022/07/31 22:36:57.228 +08:00] [INFO] [client.go:673] ["[pd] tso dispatcher created"] [dc-location=global]

4981923696 4981923392panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x12b00a6e0]
// this is error
goroutine 17 [running, locked to thread]:
main.TxnSave({0x8, 0x128f21f70, 0x14000190000?}, {0x12, 0x128f21e40, 0x12a806d38?})
    /Users/user/go-tikv/tikv.go:53 +0x180

user_1161484 · Answer

You're misunderstanding the Pointer constructor's argument. It's a native peer value, not an allocation size.

What you have:

keys.data[0] = new Pointer(byte1.length + 1);

From the Pointer Javadoc:

public Pointer(long peer)

Create from native pointer. Don't use this unless you know what you're doing.

The correct way to allocate new memory (which calls malloc() internally) is the Memory class which extends Pointer. The argument for the Memory constructor is indeed the size:

public Memory(long size)

Allocate space in the native heap via a call to C's malloc.

Parameters:

size - number of bytes of space to allocate

So you want:

keys.data[0] = new Memory(byte1.length + 1);

One other note:

Memory isn't cleared on allocation, so if you want that "+1" null terminator you need to either clear() the memory or explicitly set that null byte when writing a string to it (or perhaps append a null byte to the string in Java before converting to bytes).

Updating for your follow-on questions:

You have a mismatch between the C header (which has data first followed by len and cap and your own GoSlice structure definition which puts data in the middle. When you pass this to native, you are just sending the value of that initial long which is probably zero, and getting the nil pointer error.

Additionally, you are not allocating enough memory for what you're doing. When you declare a Structure in JNA it allocates the memory it needs. In your earlier version of code you used Structure.toArray() to allocate more memory for more structures -- this was correct (but you did it twice!). You should do that once.

In your latest code you set value as just one GoSlice structure. That memory is just 24 bytes, for the two longs and the Pointer. But then for your second array value you are using the original data (which is in the wrong order) and trying to write to a memory location that isn't even allocated.

In summary:

Declare the field order in your Java GoSlice to match the native header
Use the Structure's toArray() on an initial GoSlice structure to allocate space for an array of them.
Use the data field of the GoSlice[n] structure to write the string values to.

How can I use JNA to pass byte[][] to go

1 Answers