英文:
How can I use JNA to pass byte[][] to go
问题
这是go代码:
import (
"C"
"fmt"
)
//export Print
func Print(keys, values [][]byte) {
for len(keys) > 0 {
err := txn.Set(keys[0], values[0])
errMustBeNil(err)
fmt.Printf("%s %s", string(keys[0]), string(values[0]))
keys = keys[1:]
values = values[1:]
}
}
这是libPrint.h:
/*
static assertion to make sure the file is being used on architecture
at least with matching size of GoInt.
*/
typedef char _check_for_64_bit_pointer_matching_GoInt[sizeof(void*)==64/8 ? 1:-1];
#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef _GoString_ GoString;
#endif
typedef void *GoMap;
typedef void *GoChan;
typedef struct { void *t; void *v; } GoInterface;
typedef struct { void *data; GoInt len; GoInt cap; } GoSlice;
#endif
/* End of boilerplate cgo prologue. */
#ifdef __cplusplus
extern "C" {
#endif
extern void Print(GoSlice keys, GoSlice values);
#ifdef __cplusplus
}
#endif
这是我的Java代码,我想逐个将byte[]放入指针中,GoSlice
表示go
中的数组[],我不知道这样是否正确:
public interface Test extends Library {
Test TEST = (Test) Native.load("Print", Test.class);
void Print(GoSlice key, GoSlice val);
class GoSlice extends Structure {
public Pointer[] data;
public long len;
public long cap;
}
static void main(String[] args) {
byte[] byte1 = "key1".getBytes(StandardCharsets.UTF_8);
byte[] byte2 = "value1new".getBytes(StandardCharsets.UTF_8);
GoSlice keys = new GoSlice();
keys.data = new Pointer[1];
keys.data[0] = new Pointer(byte1.length + 1);
keys.len = 1;
keys.cap = 1;
GoSlice values = new GoSlice();
values.data = new Pointer[1];
keys.data[0] = new Pointer(byte2.length + 1);
values.len = 1;
values.cap = 1;
keys.data[0].write(0, byte1, 0, byte1.length);
values.data[0].write(0, byte2, 0, byte2.length);
Test.TEST.Print(keys, values);
}
}
但它不能正常工作,我觉得byte[][]没有正确转换为go中的类型。
这是运行Java程序后的控制台日志:
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00000001a2775a34, pid=64329, tid=0x0000000000002903
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.62.0.19-CA-macos-aarch64) (8.0_332-b09) (build 1.8.0_332-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.332-b09 mixed mode bsd-aarch64 compressed oops)
# Problematic frame:
# C [libsystem_platform.dylib+0x3a34] _platform_memmove+0xf4
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/user/project/hs_err_pid64329.log
#
# If you would like to submit a bug report, please visit:
# http://www.azul.com/support/
#
这是我的第一个更改:
public interface TxnClient extends Library {
TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);
int TxnSave(GoSlice[] key, GoSlice[] val);
class GoSlice extends Structure {
public long cap;
public Pointer data;
public long len;
@Override
protected List<String> getFieldOrder() {
return Arrays.asList("cap", "data", "len");
}
}
static void main(String[] args) {
byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);
GoSlice tmp = new GoSlice();
GoSlice[] keys = (GoSlice[]) tmp.toArray(2);
keys[0].data = new Memory(key1.length + 1);
keys[0].len = keys[0].cap = key1.length;
keys[0].data.write(0, key1, 0, key1.length);
keys[1].data = new Memory(key2.length + 1);
keys[1].len = keys[1].cap = key2.length;
keys[1].data.write(0, key2, 0, key2.length);
GoSlice[] values = (GoSlice[]) tmp.toArray(2);
values[0].data = new Memory(value1.length + 1);
values[0].len = values[0].cap = value1.length;
values[0].data.write(0, value1, 0, value1.length);
values[1].data = new Memory(value2.length + 1);
values[1].len = values[1].cap = value2.length;
values[1].data.write(0, value2, 0, value2.length);
int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
System.out.println(i);
}
}
5431788016 5431788016panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x9 pc=0x1225b2b80]
goroutine 17 [running, locked to thread]:
main.TxnSave({0x9, 0x143c281f0, 0x14000182000?}, {0x9, 0x143c281f0, 0x121daf2c8?})
/Users/user/project/print.go:52 +0x180
第二次更改:
//export TxnSave
func TxnSave(keys, values [][]byte) (res int) {
// 忽略
}
public interface TxnClient extends Library {
TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);
int TxnSave(GoSlice key, GoSlice val);
class GoSlice extends Structure {
public long cap;
public Pointer data;
public long len;
@Override
protected List<String> getFieldOrder() {
return Arrays.asList("cap", "data", "len");
}
}
static void main(String[] args) {
byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);
GoSlice keys = new GoSlice();
keys.data = new Memory(key1.length + key2.length + 1);
keys.data.write(0, key1, 0, key1.length);
keys.len = keys.cap = key1.length + key2.length;
keys.data.write(key1.length, key2, 0, key2.length);
GoSlice values = new GoSlice();
values.data = new Memory(value1.length + value2.length + 1);
values.len = values.cap = value1.length + value2.length;
values.data.write(0, value1, 0, value1.length);
values.data.write(value1.length, value2, 0, value2.length);
int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
System.out.println(i);
}
}
[2022/07/31 22:36:57.225 +08:00] [INFO] [client.go:378] ["[pd] create pd client with endpoints"] [pd-address="[127.0.0.1:2379]"]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:350] ["[pd] switch leader"] [new-leader=http://127.0.0.1:2379] [old-leader=]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:105] ["[pd] init cluster id"] [cluster-id=7126545341327379321]
[2022/07/31 22:36:57.228 +08:00] [INFO] [client.go:673] ["[pd] tso dispatcher created"] [dc-location=global]
4981923696 4981923392panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x12b00a6e0]
// 这是错误
goroutine 17 [running, locked to thread]:
main.TxnSave({0x8, 0x128f21f70, 0x14000190000?}, {0x12, 0x128f21e40, 0x12a806d38?})
/Users/user/go-tikv/tikv.go:53 +0x180
英文:
This is go code
import (
"C"
"fmt"
)
//export Print
func Print(keys, values [][]byte) {
for len(keys) > 0 {
err := txn.Set(keys[0], values[0])
errMustBeNil(err)
fmt.Printf("%s %s", string(keys[0]), string(values[0]))
keys = keys[1:]
values = values[1:]
}
}
This is libPrint.h
/*
static assertion to make sure the file is being used on architecture
at least with matching size of GoInt.
*/
typedef char _check_for_64_bit_pointer_matching_GoInt[sizeof(void*)==64/8 ? 1:-1];
#ifndef GO_CGO_GOSTRING_TYPEDEF
typedef _GoString_ GoString;
#endif
typedef void *GoMap;
typedef void *GoChan;
typedef struct { void *t; void *v; } GoInterface;
typedef struct { void *data; GoInt len; GoInt cap; } GoSlice;
#endif
/* End of boilerplate cgo prologue. */
#ifdef __cplusplus
extern "C" {
#endif
extern void Print(GoSlice keys, GoSlice values);
#ifdef __cplusplus
}
#endif
This is my Java code,I want to put byte[] into pointer one by one,GoSlice
is represents the array[] in go
,I don't know if this is correct
public interface Test extends Library {
Test TEST = (Test) Native.load("Print", Test.class);
void Print(GoSlice key, GoSlice val);
class GoSlice extends Structure {
public Pointer[] data;
public long len;
public long cap;
}
static void main(String[] args) {
byte[] byte1 = "key1".getBytes(StandardCharsets.UTF_8);
byte[] byte2 = "value1new".getBytes(StandardCharsets.UTF_8);
GoSlice keys = new GoSlice();
keys.data = new Pointer[1];
keys.data[0] = new Pointer(byte1.length + 1);
keys.len = 1;
keys.cap = 1;
GoSlice values = new GoSlice();
values.data = new Pointer[1];
keys.data[0] = new Pointer(byte2.length + 1);
values.len = 1;
values.cap = 1;
keys.data[0].write(0, byte1, 0, byte1.length);
values.data[0].write(0, byte2, 0, byte2.length);
Test.TEST.Print(keys, values);
}
}
But it doesn't work correctly,I feel that byte[][] is not correctly converted to the type in go.
> This is the console log after running the Java program
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x00000001a2775a34, pid=64329, tid=0x0000000000002903
#
# JRE version: OpenJDK Runtime Environment (Zulu 8.62.0.19-CA-macos-aarch64) (8.0_332-b09) (build 1.8.0_332-b09)
# Java VM: OpenJDK 64-Bit Server VM (25.332-b09 mixed mode bsd-aarch64 compressed oops)
# Problematic frame:
# C [libsystem_platform.dylib+0x3a34] _platform_memmove+0xf4
#
# Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again
#
# An error report file with more information is saved as:
# /Users/user/project/hs_err_pid64329.log
#
# If you would like to submit a bug report, please visit:
# http://www.azul.com/support/
#
This is my first change
public interface TxnClient extends Library {
TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);
int TxnSave(GoSlice[] key, GoSlice[] val);
class GoSlice extends Structure {
public long cap;
public Pointer data;
public long len;
@Override
protected List<String> getFieldOrder() {
return Arrays.asList("cap", "data", "len");
}
}
static void main(String[] args) {
byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);
GoSlice tmp = new GoSlice();
GoSlice[] keys = (GoSlice[]) tmp.toArray(2);
keys[0].data = new Memory(key1.length + 1);
keys[0].len = keys[0].cap = key1.length;
keys[0].data.write(0, key1, 0, key1.length);
keys[1].data = new Memory(key2.length + 1);
keys[1].len = keys[1].cap = key2.length;
keys[1].data.write(0, key2, 0, key2.length);
GoSlice[] values = (GoSlice[]) tmp.toArray(2);
values[0].data = new Memory(value1.length + 1);
values[0].len = values[0].cap = value1.length;
values[0].data.write(0, value1, 0, value1.length);
values[1].data = new Memory(value2.length + 1);
values[1].len = values[1].cap = value2.length;
values[1].data.write(0, value2, 0, value2.length);
int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
System.out.println(i);
}
}
5431788016 5431788016panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x9 pc=0x1225b2b80]
goroutine 17 [running, locked to thread]:
main.TxnSave({0x9, 0x143c281f0, 0x14000182000?}, {0x9, 0x143c281f0, 0x121daf2c8?})
/Users/user/project/print.go:52 +0x180
second change
//export TxnSave
func TxnSave(keys, values [][]byte) (res int) {
// ignore
}
public interface TxnClient extends Library {
TxnClient TNX_CLIENT = (TxnClient) Native.load("Tikv", TxnClient.class);
int TxnSave(GoSlice key, GoSlice val);
class GoSlice extends Structure {
public long cap;
public Pointer data;
public long len;
@Override
protected List<String> getFieldOrder() {
return Arrays.asList("cap", "data", "len");
}
}
static void main(String[] args) {
byte[] key1 = "key1".getBytes(StandardCharsets.UTF_8);
byte[] value1 = "value1new".getBytes(StandardCharsets.UTF_8);
byte[] key2 = "key2".getBytes(StandardCharsets.UTF_8);
byte[] value2 = "value2new".getBytes(StandardCharsets.UTF_8);
GoSlice keys = new GoSlice();
keys.data = new Memory(key1.length + key2.length + 1);
keys.data.write(0, key1, 0, key1.length);
keys.len = keys.cap = key1.length + key2.length;
keys.data.write(key1.length, key2, 0, key2.length);
GoSlice values = new GoSlice();
values.data = new Memory(value1.length + value2.length + 1);
values.len = values.cap = value1.length + value2.length;
values.data.write(0, value1, 0, value1.length);
values.data.write(value1.length, value2, 0, value2.length);
int i = TxnClient.TNX_CLIENT.TxnSave(keys, values);
System.out.println(i);
}
}
[2022/07/31 22:36:57.225 +08:00] [INFO] [client.go:378] ["[pd] create pd client with endpoints"] [pd-address="[127.0.0.1:2379]"]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:350] ["[pd] switch leader"] [new-leader=http://127.0.0.1:2379] [old-leader=]
[2022/07/31 22:36:57.228 +08:00] [INFO] [base_client.go:105] ["[pd] init cluster id"] [cluster-id=7126545341327379321]
[2022/07/31 22:36:57.228 +08:00] [INFO] [client.go:673] ["[pd] tso dispatcher created"] [dc-location=global]
4981923696 4981923392panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x2 addr=0x8 pc=0x12b00a6e0]
// this is error
goroutine 17 [running, locked to thread]:
main.TxnSave({0x8, 0x128f21f70, 0x14000190000?}, {0x12, 0x128f21e40, 0x12a806d38?})
/Users/user/go-tikv/tikv.go:53 +0x180
答案1
得分: 1
你误解了Pointer
构造函数的参数。它是一个本地对等值,而不是分配的大小。
你现在的代码是这样的:
keys.data[0] = new Pointer(byte1.length + 1);
根据Pointer
的Javadoc:
public Pointer(long peer)
从本地指针创建。除非你知道自己在做什么,否则不要使用此方法。
正确的分配新内存的方式(内部调用malloc()
)是使用扩展了Pointer
的Memory
类。Memory
构造函数的参数确实是大小:
public Memory(long size)
通过调用C的malloc在本地堆中分配空间。
参数:
size - 要分配的空间的字节数
所以你应该这样写:
keys.data[0] = new Memory(byte1.length + 1);
还有一点需要注意:
Memory
在分配时不会被清除,所以如果你想要那个“+1”的空字符终止符,你需要在将字符串写入内存时要么使用clear()
方法清除内存,要么在将字符串转换为字节之前在Java中显式地设置那个空字节(或者在Java中将空字节附加到字符串)。
针对你的后续问题进行更新:
你的C头文件中的定义(data
在前,然后是len
和cap
)与你自己的GoSlice
结构定义不匹配。当你将其传递给本地代码时,实际上只是发送了那个初始long
的值,这个值可能是零,所以会出现nil
指针错误。
此外,你为你的操作分配的内存不足。当你在JNA中声明一个Structure
时,它会分配所需的内存。在你之前的代码版本中,你使用了Structure.toArray()
来为更多的结构分配更多的内存,这是正确的(但你做了两次!)。你应该只做一次。
在你最新的代码中,你将value
设置为一个GoSlice
结构。该内存只有24个字节,用于两个long
和Pointer
。但是对于第二个数组值,你使用了原始的data
(顺序错误),并尝试写入一个甚至没有分配的内存位置。
总结一下:
- 在Java的
GoSlice
中声明字段顺序以匹配本地头文件。 - 对初始的
GoSlice
结构使用Structure的toArray()
方法来为它们的数组分配空间。 - 使用GoSlice[n]结构的
data
字段来写入字符串值。
英文:
You're misunderstanding the Pointer
constructor's argument. It's a native peer value, not an allocation size.
What you have:
keys.data[0] = new Pointer(byte1.length + 1);
From the Pointer
Javadoc:
> public Pointer(long peer)
>
> Create from native pointer. Don't use this unless you know what you're doing.
The correct way to allocate new memory (which calls malloc()
internally) is the Memory
class which extends Pointer
. The argument for the Memory
constructor is indeed the size:
> public Memory(long size)
>
> Allocate space in the native heap via a call to C's malloc.
>
> Parameters:
>
> size - number of bytes of space to allocate
So you want:
keys.data[0] = new Memory(byte1.length + 1);
One other note:
Memory
isn't cleared on allocation, so if you want that "+1" null terminator you need to either clear()
the memory or explicitly set that null byte when writing a string to it (or perhaps append a null byte to the string in Java before converting to bytes).
Updating for your follow-on questions:
You have a mismatch between the C header (which has data
first followed by len
and cap
and your own GoSlice
structure definition which puts data
in the middle. When you pass this to native, you are just sending the value of that initial long
which is probably zero, and getting the nil
pointer error.
Additionally, you are not allocating enough memory for what you're doing. When you declare a Structure
in JNA it allocates the memory it needs. In your earlier version of code you used Structure.toArray()
to allocate more memory for more structures -- this was correct (but you did it twice!). You should do that once.
In your latest code you set value
as just one GoSlice
structure. That memory is just 24 bytes, for the two long
s and the Pointer
. But then for your second array value you are using the original data
(which is in the wrong order) and trying to write to a memory location that isn't even allocated.
In summary:
- Declare the field order in your Java
GoSlice
to match the native header - Use the Structure's
toArray()
on an initial GoSlice structure to allocate space for an array of them. - Use the
data
field of the GoSlice[n] structure to write the string values to.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论