英文:
Is there a graceful way to pass large byte slice or io.Reader to c through cgo without memory copy?
问题
我有一个需要大量数据的C程序。
假设我在Go语言中有一个4GB的字节切片，刚刚从一个大文件、互联网或套接字中加载进来。
然后我想将它传递给cgo。如果我传递一个指向字节缓冲区的指针，那么缓冲区仍然在Go语言中，因为缓冲区可能会被垃圾回收，从而在C端创建段错误。而且io.Reader不能简单地传递给cgo，因为它太通用了。
我知道我可以传递文件路径，或者在C中编写互联网请求代码来避免使用cgo传递字节。但这不方便，我仍然希望保持使用Go语言的便利性。
那么我们如何优雅地处理cgo中的大字节传递呢？如果没有其他选择，我想确保没有其他选择。
英文:
I have a c program that needs a lot of data to work.
Suppose I have a byte slice of 4GB in go, just loaded from a large file, or internet, or socket.
Then I want to pass it to cgo. If I pass a pointer to the byte buffer, the buffer is still in go, because the buffer can be garbage collected, and thus create segment fault in C side. And io.Reader cannot be simply passed to cgo beacuse it's too general.
I know I can pass the filepath, or write internet request code in C to avoid a cgo bytes pass. But that is not convenient, I still want to keep the advantage of convenience of golang.
So how can we gracefully handle large bytes pass in cgo? If there is no another option, I want to make sure there is no another option.
答案1
得分: 1
如果我传递一个指向字节缓冲区的指针，那么缓冲区仍然在Go语言中，因为缓冲区可能会被垃圾回收，从而在C语言中导致段错误。
只要C函数在调用返回后不再引用该指针，你是可以安全地将Go指针传递给C的。Go编译器会在C调用的持续时间内将分配固定在原地。
详细信息请参考https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers。
英文:
> If I pass a pointer to the byte buffer, the buffer is still in go, because the buffer can be garbage collected, and thus create segment fault in C side.
You can safely pass a Go pointer to C, as long as the C function does not refer to that pointer any more after the call returns. The Go compiler will pin the allocation in place for the duration of the C call.
See https://pkg.go.dev/cmd/cgo#hdr-Passing_pointers for the precise details.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论