CGO:将Go字节数组转换为C的char*,并在字节数组中遇到空终止符的问题。

huangapple go评论89阅读模式
英文:

CGO: converting Go byte array into C char* and back, issue with null terminator in byte array

问题

简要背景:
我正在为我的公司创建一个从uint64到[]byte的cgo缓存。由于[]byte被视为指针,Golang的map[uint64][]byte会因垃圾回收器而产生相当大的延迟。因此,我想尝试使用CGO来避免这个问题。我目前正在实现一个C++ unordered_map<unsigned long long, char*>来处理这个问题。我成功地让C++包装器工作了,但是我遇到了一个重大问题。

目前,我正在使用以下方式将我的Go字节数组转换:

b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))

然而,结果是我的bb是[]byte{1}。字节数组中的0被视为'/0',从而缩短了字符串。此外,当我使用delete (map->find(key))->second删除条目时,似乎引发了内存不足的问题。我怀疑这是因为第一个'/0'之后的字符没有被释放。我不确定还有其他方法可以解决这个问题。就个人而言,我对CGO还很陌生,所以在这个项目之前我从未使用过它,所以任何帮助都将不胜感激。

英文:

Brief Context:
I am trying to create a cgo cache from uint64->[]byte for my company. Golang's map[uint64][]byte incur considerable latency due to the garbage collector as []byte is considered a pointer. As such, I would like to try to use CGO to avoid the issue. I am currently implementing a C++ unordered_map&lt;unsigned long long, char*&gt; to handle that. I managed to get the C++ wrappers to work but I am facing a major issue.

Currently, I am converting my Go byte array using

b := []byte{1,0,3,32,2,2,2,2}
str := C.String(b)
bb := []byte(C.GoString(str))

However, it turns out my my bb is []byte{1}. The 0 in the byte array is seen as the &#39;/0&#39; and thus shorten the string. Furthermore, it seems to have cause out of memory issue when I delete entries with

delete (map-&gt;find(key))-&gt;second. I suspect this is because that chars after the first &#39;/0&#39; does not get deallocated.
I am not sure how else to do this. Personally, I am new to CGO so I never used it prior to this project so any help would be appreciated.

答案1

得分: 4

有两个问题:

  • 使用C.CBytes而不是C.CString

    在使用cgo之前,请完整阅读这个

  • C.C*函数会分配内存(C.CStringC.CBytes会这样做),它们在内部调用链接的libc库中的malloc()来获取目标内存块,你应该最终调用C.Free()(它调用libcfree())来释放它们,正如文档中所述。

    据我所知,C++编译器并不一定要使用libcmalloc()free()来实现newdelete,所以在C.C*函数的结果上调用delete是一条通往灾难的确定路径。

    在我看来,最简单的解决方法是从你的C++端导出一个“构造函数”函数,类似于:

    extern "C" {
        char* clone(char *src, size_t len);
    }
    

    …它将:1)使用最适合C++的方法分配一个长度为len的内存块;2)将len字节从src复制到它;3)返回它。

    然后你可以从Go端调用它,如C.clone(&b[0], len(b)),然后结束:C++端可以自由地在结果上调用delete

英文:

There are two problems:

  • Use C.CBytes, not C.CString.

    Please read this in its entirety before using cgo.

  • The C.C* functions which allocate (C.CString and C.CBytes do that) internally call malloc() from the linked in libc library to get the destination memory block, and you're supposed to eventually call C.Free() on them (which calls libc's free()), as documented.

    AFAIK, C++ compiler is by no means oblidged to use libc's malloc() and free() to implement new and delete, so calling delete on the results of C.C* functions is a sure path to disaster.

    The simplest way to solve this, IMO, is to export a "constructor" function from your C++ side: something like

    extern &quot;C&quot; {
        char* clone(char *src, size_t len);
    }
    

    …which would 1) allocate a memory block of length len using whatever method works best for C++; 2) copy len bytes from src to it; 3) return it.

    You could then call it from the Go side—as C.clone(&amp;b[0], len(b)) and call it a day: the C++ side is free to call delete on the result.

huangapple
  • 本文由 发表于 2021年7月15日 17:38:35
  • 转载请务必保留本文链接:https://go.coder-hub.com/68391408.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定