How can I use JNA to pass byte[][] to go

huangapple go评论104阅读模式
英文:

How can I use JNA to pass byte[][] to go

问题

这是go代码:

import (
	"C"
	"fmt"
)

//export Print
func Print(keys, values [][]byte) {
	for len(keys) > 0 {
		err := txn.Set(keys[0], values[0])
		errMustBeNil(err)
		fmt.Printf("%s %s", string(keys[0]), string(values[0]))
		keys = keys[1:]
		values = values[1:]
	}
}

这是libPrint.h

/*
  static assertion to make sure the file is being used on architecture
  at least with matching size of GoInt.
*/
typedef char _check_for_64_bit_pointer_matching_GoInt[sizeof(void*)==64/8 ? 1:-1];

#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef _GoString_ GoString;
#endif
typedef void *GoMap;
typedef void *GoChan;
typedef struct { void *t; void *v; } GoInterface;
typedef struct { void *data; GoInt len; GoInt cap; } GoSlice;

#endif

/* End of boilerplate cgo prologue.  */

#ifdef __cplusplus
extern "C" {
#endif

extern void Print(GoSlice keys, GoSlice values);

#ifdef __cplusplus
}
#endif

这是我的Java代码,我想逐个将byte[]放入指针中,GoSlice表示go中的数组[],我不知道这样是否正确:

public interface Test extends Library {
  Test TEST = (Test) Native.load("Print", Test.class);

  void Print(GoSlice key, GoSlice val);

  class GoSlice extends Structure {
    public Pointer[] data;
    public long len;
    public long cap;
  }

  static void main(String[] args) {
    byte[] byte1 = "key1".getBytes(StandardCharsets.UTF_8);
    byte[] byte2 = "value1new".getBytes(StandardCharsets.UTF_8);

    GoSlice keys = new GoSlice();
    keys.data = new Pointer[1];
    keys.data[0] = new Pointer(byte1.length + 1);
    keys.len = 1;
    keys.cap = 1;

    GoSlice values = new GoSlice();
    values.data = new Pointer[1];
    keys.data[0] = new Pointer(byte2.length + 1);
    values.len = 1;
    values.cap = 1;

    keys.data[0].write(0, byte1, 0, byte1.length);
    values.data[0].write(0, byte2, 0, byte2.length);

    Test.TEST.Print(keys, values);
  }
}

但它不能正常工作,我觉得byte[][]没有正确转换为go中的类型。

这是运行Java程序后的控制台日志:

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00000001a2775a34, pid=64329, tid=0x0000000000002903
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.62.0.19-CA-macos-aarch64) (8.0_332-b09) (build 1.8.0_332-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.332-b09 mixed mode bsd-aarch64 compressed oops)
# Problematic frame:
# C  [libsystem_platform.dylib+0x3a34]  _platform_memmove+0xf4
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/user/project/hs_err_pid64329.log
#
# If you would like to submit a bug report, please visit:
#   http://www.azul.com/support/
#

这是我的第一个更改

public interface TxnClient extends Library {
  TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);

  int TxnSave(GoSlice[] key, GoSlice[] val);

  class GoSlice extends Structure {
    public long cap;
    public Pointer data;
    public long len;

    @Override
    protected List<String> getFieldOrder() {
      return Arrays.asList("cap", "data", "len");
    }
  }

  static void main(String[] args) {
    byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
    byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
    byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
    byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);

    GoSlice tmp = new GoSlice();
    GoSlice[] keys = (GoSlice[]) tmp.toArray(2);
    keys[0].data = new Memory(key1.length + 1);
    keys[0].len = keys[0].cap = key1.length;
    keys[0].data.write(0, key1, 0, key1.length);

    keys[1].data = new Memory(key2.length + 1);
    keys[1].len = keys[1].cap = key2.length;
    keys[1].data.write(0, key2, 0, key2.length);

    GoSlice[] values = (GoSlice[]) tmp.toArray(2);
    values[0].data = new Memory(value1.length + 1);
    values[0].len = values[0].cap = value1.length;
    values[0].data.write(0, value1, 0, value1.length);

    values[1].data = new Memory(value2.length + 1);
    values[1].len = values[1].cap = value2.length;
    values[1].data.write(0, value2, 0, value2.length);

    int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
    System.out.println(i);
  }
}
5431788016 5431788016panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x9 pc=0x1225b2b80]
goroutine 17 [running, locked to thread]:
main.TxnSave({0x9, 0x143c281f0, 0x14000182000?}, {0x9, 0x143c281f0, 0x121daf2c8?})
/Users/user/project/print.go:52 +0x180

第二次更改

//export TxnSave
func TxnSave(keys, values [][]byte) (res int) {
	// 忽略
}
public interface TxnClient extends Library {
  TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);

  int TxnSave(GoSlice key, GoSlice val);

  class GoSlice extends Structure {
    public long cap;
    public Pointer data;
    public long len;

    @Override
    protected List<String> getFieldOrder() {
      return Arrays.asList("cap", "data", "len");
    }
  }

  static void main(String[] args) {
    byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
    byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
    byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
    byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);

    GoSlice keys = new GoSlice();
    keys.data = new Memory(key1.length + key2.length + 1);
    keys.data.write(0, key1, 0, key1.length);
    keys.len = keys.cap = key1.length + key2.length;
    keys.data.write(key1.length, key2, 0, key2.length);

    GoSlice values = new GoSlice();
    values.data = new Memory(value1.length + value2.length + 1);
    values.len = values.cap = value1.length + value2.length;
    values.data.write(0, value1, 0, value1.length);
    values.data.write(value1.length, value2, 0, value2.length);

    int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
    System.out.println(i);
  }
}
[2022/07/31 22:36:57.225 +08:00] [INFO] [client.go:378] ["[pd] create pd client with endpoints"] [pd-address="[127.0.0.1:2379]"]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:350] ["[pd] switch leader"] [new-leader=http://127.0.0.1:2379] [old-leader=]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:105] ["[pd] init cluster id"] [cluster-id=7126545341327379321]
[2022/07/31 22:36:57.228 +08:00] [INFO] [client.go:673] ["[pd] tso dispatcher created"] [dc-location=global]
4981923696 4981923392panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x12b00a6e0]
// 这是错误
goroutine 17 [running, locked to thread]:
main.TxnSave({0x8, 0x128f21f70, 0x14000190000?}, {0x12, 0x128f21e40, 0x12a806d38?})
/Users/user/go-tikv/tikv.go:53 +0x180
英文:

This is go code

import (
	&quot;C&quot;
	&quot;fmt&quot;
)
//export Print
func Print(keys, values [][]byte) {
  for len(keys) &gt; 0 {
  err := txn.Set(keys[0], values[0])
  errMustBeNil(err)
  fmt.Printf(&quot;%s %s&quot;, string(keys[0]), string(values[0]))
  keys = keys[1:]
  values = values[1:]
 }
}

This is libPrint.h

/*
  static assertion to make sure the file is being used on architecture
  at least with matching size of GoInt.
*/
typedef char _check_for_64_bit_pointer_matching_GoInt[sizeof(void*)==64/8 ? 1:-1];

#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef _GoString_ GoString;
#endif
typedef void *GoMap;
typedef void *GoChan;
typedef struct { void *t; void *v; } GoInterface;
typedef struct { void *data; GoInt len; GoInt cap; } GoSlice;

#endif

/* End of boilerplate cgo prologue.  */

#ifdef __cplusplus
extern &quot;C&quot; {
#endif

extern void Print(GoSlice keys, GoSlice values);

#ifdef __cplusplus
}
#endif

This is my Java code,I want to put byte[] into pointer one by one,GoSlice is represents the array[] in go,I don't know if this is correct

public interface Test extends Library {
  Test TEST = (Test) Native.load(&quot;Print&quot;, Test.class);

  void Print(GoSlice key, GoSlice val);

  class GoSlice extends Structure {
    public Pointer[] data;
    public long len;
    public long cap;
  }
  static void main(String[] args) {
    byte[] byte1 = &quot;key1&quot;.getBytes(StandardCharsets.UTF_8);
    byte[] byte2 = &quot;value1new&quot;.getBytes(StandardCharsets.UTF_8);

    GoSlice keys = new GoSlice();
    keys.data = new Pointer[1];
    keys.data[0] = new Pointer(byte1.length + 1);
    keys.len = 1;
    keys.cap = 1;

    GoSlice values = new GoSlice();
    values.data = new Pointer[1];
    keys.data[0] = new Pointer(byte2.length + 1);
    values.len = 1;
    values.cap = 1;

    keys.data[0].write(0, byte1, 0, byte1.length);
    values.data[0].write(0, byte2, 0, byte2.length);

    Test.TEST.Print(keys, values);
  }
}

But it doesn't work correctly,I feel that byte[][] is not correctly converted to the type in go.

> This is the console log after running the Java program

#
# A fatal error has been detected by the Java Runtime Environment:
#
#  SIGSEGV (0xb) at pc=0x00000001a2775a34, pid=64329, tid=0x0000000000002903
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.62.0.19-CA-macos-aarch64) (8.0_332-b09) (build 1.8.0_332-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.332-b09 mixed mode bsd-aarch64 compressed oops)
# Problematic frame:
# C  [libsystem_platform.dylib+0x3a34]  _platform_memmove+0xf4
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try &quot;ulimit -c unlimited&quot; before starting Java again
#
# An error report file with more information is saved as:
# /Users/user/project/hs_err_pid64329.log
#
# If you would like to submit a bug report, please visit:
#   http://www.azul.com/support/
#

This is my first change

public interface TxnClient extends Library {
  TxnClient TNX_CLIENT = (TxnClient) Native.load(&quot;Tikv&quot;, TxnClient.class);

  int TxnSave(GoSlice[] key, GoSlice[] val);

  class GoSlice extends Structure {
    public long cap;
    public Pointer data;
    public long len;

    @Override
    protected List&lt;String&gt; getFieldOrder() {
      return Arrays.asList(&quot;cap&quot;, &quot;data&quot;, &quot;len&quot;);
    }
  }

  static void main(String[] args) {
    byte[] key1 = &quot;key1&quot;.getBytes(StandardCharsets.UTF_8);
    byte[] value1 = &quot;value1new&quot;.getBytes(StandardCharsets.UTF_8);
    byte[] key2 = &quot;key2&quot;.getBytes(StandardCharsets.UTF_8);
    byte[] value2 = &quot;value2new&quot;.getBytes(StandardCharsets.UTF_8);

    GoSlice tmp = new GoSlice();
    GoSlice[] keys = (GoSlice[]) tmp.toArray(2);
    keys[0].data = new Memory(key1.length + 1);
    keys[0].len = keys[0].cap = key1.length;
    keys[0].data.write(0, key1, 0, key1.length);

    keys[1].data = new Memory(key2.length + 1);
    keys[1].len = keys[1].cap = key2.length;
    keys[1].data.write(0, key2, 0, key2.length);

    GoSlice[] values = (GoSlice[]) tmp.toArray(2);
    values[0].data = new Memory(value1.length + 1);
    values[0].len = values[0].cap = value1.length;
    values[0].data.write(0, value1, 0, value1.length);

    values[1].data = new Memory(value2.length + 1);
    values[1].len = values[1].cap = value2.length;
    values[1].data.write(0, value2, 0, value2.length);

    int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
    System.out.println(i);
  }
}
5431788016 5431788016panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x9 pc=0x1225b2b80]
goroutine 17 [running, locked to thread]:
main.TxnSave({0x9, 0x143c281f0, 0x14000182000?}, {0x9, 0x143c281f0, 0x121daf2c8?})
/Users/user/project/print.go:52 +0x180

second change

//export TxnSave
func TxnSave(keys, values [][]byte) (res int) {
 // ignore
}
public interface TxnClient extends Library {
  TxnClient TNX_CLIENT = (TxnClient) Native.load(&quot;Tikv&quot;, TxnClient.class);

  int TxnSave(GoSlice key, GoSlice val);

  class GoSlice extends Structure {
    public long cap;
    public Pointer data;
    public long len;

    @Override
    protected List&lt;String&gt; getFieldOrder() {
      return Arrays.asList(&quot;cap&quot;, &quot;data&quot;, &quot;len&quot;);
    }
  }

  static void main(String[] args) {
    byte[] key1 = &quot;key1&quot;.getBytes(StandardCharsets.UTF_8);
    byte[] value1 = &quot;value1new&quot;.getBytes(StandardCharsets.UTF_8);
    byte[] key2 = &quot;key2&quot;.getBytes(StandardCharsets.UTF_8);
    byte[] value2 = &quot;value2new&quot;.getBytes(StandardCharsets.UTF_8);

    GoSlice keys = new GoSlice();
    keys.data = new Memory(key1.length + key2.length + 1);
    keys.data.write(0, key1, 0, key1.length);
    keys.len = keys.cap = key1.length + key2.length;
    keys.data.write(key1.length, key2, 0, key2.length);

    GoSlice values = new GoSlice();
    values.data = new Memory(value1.length + value2.length + 1);
    values.len = values.cap = value1.length + value2.length;
    values.data.write(0, value1, 0, value1.length);
    values.data.write(value1.length, value2, 0, value2.length);

    int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
    System.out.println(i);
  }
}
[2022/07/31 22:36:57.225 +08:00] [INFO] [client.go:378] [&quot;[pd] create pd client with endpoints&quot;] [pd-address=&quot;[127.0.0.1:2379]&quot;]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:350] [&quot;[pd] switch leader&quot;] [new-leader=http://127.0.0.1:2379] [old-leader=]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:105] [&quot;[pd] init cluster id&quot;] [cluster-id=7126545341327379321]
[2022/07/31 22:36:57.228 +08:00] [INFO] [client.go:673] [&quot;[pd] tso dispatcher created&quot;] [dc-location=global]
4981923696 4981923392panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x12b00a6e0]
// this is error
goroutine 17 [running, locked to thread]:
main.TxnSave({0x8, 0x128f21f70, 0x14000190000?}, {0x12, 0x128f21e40, 0x12a806d38?})
/Users/user/go-tikv/tikv.go:53 +0x180

答案1

得分: 1

你误解了Pointer构造函数的参数。它是一个本地对等值,而不是分配的大小。

你现在的代码是这样的:

keys.data[0] = new Pointer(byte1.length + 1);

根据PointerJavadoc

public Pointer(long peer)

从本地指针创建。除非你知道自己在做什么,否则不要使用此方法。

正确的分配新内存的方式(内部调用malloc())是使用扩展了PointerMemory类。Memory构造函数的参数确实是大小

public Memory(long size)

通过调用C的malloc在本地堆中分配空间。

参数:

size - 要分配的空间的字节数

所以你应该这样写:

keys.data[0] = new Memory(byte1.length + 1);

还有一点需要注意:

Memory在分配时不会被清除,所以如果你想要那个“+1”的空字符终止符,你需要在将字符串写入内存时要么使用clear()方法清除内存,要么在将字符串转换为字节之前在Java中显式地设置那个空字节(或者在Java中将空字节附加到字符串)。

针对你的后续问题进行更新:

你的C头文件中的定义(data在前,然后是lencap)与你自己的GoSlice结构定义不匹配。当你将其传递给本地代码时,实际上只是发送了那个初始long的值,这个值可能是零,所以会出现nil指针错误。

此外,你为你的操作分配的内存不足。当你在JNA中声明一个Structure时,它会分配所需的内存。在你之前的代码版本中,你使用了Structure.toArray()来为更多的结构分配更多的内存,这是正确的(但你做了两次!)。你应该只做一次。

在你最新的代码中,你将value设置为一个GoSlice结构。该内存只有24个字节,用于两个longPointer。但是对于第二个数组值,你使用了原始的data(顺序错误),并尝试写入一个甚至没有分配的内存位置。

总结一下:

  1. 在Java的GoSlice中声明字段顺序以匹配本地头文件。
  2. 对初始的GoSlice结构使用Structure的toArray()方法来为它们的数组分配空间。
  3. 使用GoSlice[n]结构的data字段来写入字符串值。
英文:

You're misunderstanding the Pointer constructor's argument. It's a native peer value, not an allocation size.

What you have:

keys.data[0] = new Pointer(byte1.length + 1);

From the Pointer Javadoc:
> public Pointer(long peer)
>
> Create from native pointer. Don't use this unless you know what you're doing.

The correct way to allocate new memory (which calls malloc() internally) is the Memory class which extends Pointer. The argument for the Memory constructor is indeed the size:
> public Memory(long size)
>
> Allocate space in the native heap via a call to C's malloc.
>
> Parameters:
>
> size - number of bytes of space to allocate

So you want:

keys.data[0] = new Memory(byte1.length + 1);

One other note:

Memory isn't cleared on allocation, so if you want that "+1" null terminator you need to either clear() the memory or explicitly set that null byte when writing a string to it (or perhaps append a null byte to the string in Java before converting to bytes).

Updating for your follow-on questions:

You have a mismatch between the C header (which has data first followed by len and cap and your own GoSlice structure definition which puts data in the middle. When you pass this to native, you are just sending the value of that initial long which is probably zero, and getting the nil pointer error.

Additionally, you are not allocating enough memory for what you're doing. When you declare a Structure in JNA it allocates the memory it needs. In your earlier version of code you used Structure.toArray() to allocate more memory for more structures -- this was correct (but you did it twice!). You should do that once.

In your latest code you set value as just one GoSlice structure. That memory is just 24 bytes, for the two longs and the Pointer. But then for your second array value you are using the original data (which is in the wrong order) and trying to write to a memory location that isn't even allocated.

In summary:

  1. Declare the field order in your Java GoSlice to match the native header
  2. Use the Structure's toArray() on an initial GoSlice structure to allocate space for an array of them.
  3. Use the data field of the GoSlice[n] structure to write the string values to.

huangapple
  • 本文由 发表于 2022年7月31日 14:24:17
  • 转载请务必保留本文链接:https://go.coder-hub.com/73181105.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定