英文:
CGO: converting Go byte array into C char* and back, issue with null terminator in byte array
问题
简要背景:
我正在为我的公司创建一个从uint64到[]byte的cgo缓存。由于[]byte被视为指针,Golang的map[uint64][]byte会因垃圾回收器而产生相当大的延迟。因此,我想尝试使用CGO来避免这个问题。我目前正在实现一个C++ unordered_map<unsigned long long, char*>来处理这个问题。我成功地让C++包装器工作了,但是我遇到了一个重大问题。
目前,我正在使用以下方式将我的Go字节数组转换:
b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))
然而,结果是我的bb是[]byte{1}。字节数组中的0被视为'/0',从而缩短了字符串。此外,当我使用delete (map->find(key))->second删除条目时,似乎引发了内存不足的问题。我怀疑这是因为第一个'/0'之后的字符没有被释放。我不确定还有其他方法可以解决这个问题。就个人而言,我对CGO还很陌生,所以在这个项目之前我从未使用过它,所以任何帮助都将不胜感激。
英文:
Brief Context:
I am trying to create a cgo cache from uint64->[]byte for my company. Golang's map[uint64][]byte incur considerable latency due to the garbage collector as []byte is considered a pointer. As such, I would like to try to use CGO to avoid the issue. I am currently implementing a C++ unordered_map<unsigned long long, char*> to handle that. I managed to get the C++ wrappers to work but I am facing a major issue.
Currently, I am converting my Go byte array using
b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))
However, it turns out my my bb is []byte{1}. The 0 in the byte array is seen as the '/0' and thus shorten the string. Furthermore, it seems to have cause out of memory issue when I delete entries with
delete (map->find(key))->second. I suspect this is because that chars after the first '/0' does not get deallocated.
I am not sure how else to do this. Personally, I am new to CGO so I never used it prior to this project so any help would be appreciated.
答案1
得分: 4
有两个问题:
-
使用
C.CBytes而不是C.CString。在使用
cgo之前,请完整阅读这个。 -
C.C*函数会分配内存(C.CString和C.CBytes会这样做),它们在内部调用链接的libc库中的malloc()来获取目标内存块,你应该最终调用C.Free()(它调用libc的free())来释放它们,正如文档中所述。据我所知,C++编译器并不一定要使用
libc的malloc()和free()来实现new和delete,所以在C.C*函数的结果上调用delete是一条通往灾难的确定路径。在我看来,最简单的解决方法是从你的C++端导出一个“构造函数”函数,类似于:
extern "C" { char* clone(char *src, size_t len); }…它将:1)使用最适合C++的方法分配一个长度为
len的内存块;2)将len字节从src复制到它;3)返回它。然后你可以从Go端调用它,如
C.clone(&b[0], len(b)),然后结束:C++端可以自由地在结果上调用delete。
英文:
There are two problems:
-
Use
C.CBytes, notC.CString.Please read this in its entirety before using
cgo. -
The
C.C*functions which allocate (C.CStringandC.CBytesdo that) internally callmalloc()from the linked inlibclibrary to get the destination memory block, and you're supposed to eventually callC.Free()on them (which callslibc'sfree()), as documented.AFAIK, C++ compiler is by no means oblidged to use
libc'smalloc()andfree()to implementnewanddelete, so callingdeleteon the results ofC.C*functions is a sure path to disaster.The simplest way to solve this, IMO, is to export a "constructor" function from your C++ side: something like
extern "C" { char* clone(char *src, size_t len); }…which would 1) allocate a memory block of length
lenusing whatever method works best for C++; 2) copylenbytes fromsrcto it; 3) return it.You could then call it from the Go side—as
C.clone(&b[0], len(b))and call it a day: the C++ side is free to calldeleteon the result.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。


评论