英文:
CGO: converting Go byte array into C char* and back, issue with null terminator in byte array
问题
简要背景:
我正在为我的公司创建一个从uint64到[]byte的cgo缓存。由于[]byte被视为指针,Golang的map[uint64][]byte
会因垃圾回收器而产生相当大的延迟。因此,我想尝试使用CGO来避免这个问题。我目前正在实现一个C++ unordered_map<unsigned long long, char*>
来处理这个问题。我成功地让C++包装器工作了,但是我遇到了一个重大问题。
目前,我正在使用以下方式将我的Go字节数组转换:
b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))
然而,结果是我的bb是[]byte{1}
。字节数组中的0被视为'/0'
,从而缩短了字符串。此外,当我使用delete (map->find(key))->second
删除条目时,似乎引发了内存不足的问题。我怀疑这是因为第一个'/0'
之后的字符没有被释放。我不确定还有其他方法可以解决这个问题。就个人而言,我对CGO还很陌生,所以在这个项目之前我从未使用过它,所以任何帮助都将不胜感激。
英文:
Brief Context:
I am trying to create a cgo cache from uint64->[]byte for my company. Golang's map[uint64][]byte
incur considerable latency due to the garbage collector as []byte is considered a pointer. As such, I would like to try to use CGO to avoid the issue. I am currently implementing a C++ unordered_map<unsigned long long, char*>
to handle that. I managed to get the C++ wrappers to work but I am facing a major issue.
Currently, I am converting my Go byte array using
b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))
However, it turns out my my bb is []byte{1}
. The 0 in the byte array is seen as the '/0'
and thus shorten the string. Furthermore, it seems to have cause out of memory issue when I delete entries with
delete (map->find(key))->second
. I suspect this is because that chars after the first '/0'
does not get deallocated.
I am not sure how else to do this. Personally, I am new to CGO so I never used it prior to this project so any help would be appreciated.
答案1
得分: 4
有两个问题:
-
使用
C.CBytes
而不是C.CString
。在使用
cgo
之前,请完整阅读这个。 -
C.C*
函数会分配内存(C.CString
和C.CBytes
会这样做),它们在内部调用链接的libc
库中的malloc()
来获取目标内存块,你应该最终调用C.Free()
(它调用libc
的free()
)来释放它们,正如文档中所述。据我所知,C++编译器并不一定要使用
libc
的malloc()
和free()
来实现new
和delete
,所以在C.C*
函数的结果上调用delete
是一条通往灾难的确定路径。在我看来,最简单的解决方法是从你的C++端导出一个“构造函数”函数,类似于:
extern "C" { char* clone(char *src, size_t len); }
…它将:1)使用最适合C++的方法分配一个长度为
len
的内存块;2)将len
字节从src
复制到它;3)返回它。然后你可以从Go端调用它,如
C.clone(&b[0], len(b))
,然后结束:C++端可以自由地在结果上调用delete
。
英文:
There are two problems:
-
Use
C.CBytes
, notC.CString
.Please read this in its entirety before using
cgo
. -
The
C.C*
functions which allocate (C.CString
andC.CBytes
do that) internally callmalloc()
from the linked inlibc
library to get the destination memory block, and you're supposed to eventually callC.Free()
on them (which callslibc
'sfree()
), as documented.AFAIK, C++ compiler is by no means oblidged to use
libc
'smalloc()
andfree()
to implementnew
anddelete
, so callingdelete
on the results ofC.C*
functions is a sure path to disaster.The simplest way to solve this, IMO, is to export a "constructor" function from your C++ side: something like
extern "C" { char* clone(char *src, size_t len); }
…which would 1) allocate a memory block of length
len
using whatever method works best for C++; 2) copylen
bytes fromsrc
to it; 3) return it.You could then call it from the Go side—as
C.clone(&b[0], len(b))
and call it a day: the C++ side is free to calldelete
on the result.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论