如何处理返回 CString 时的内存泄漏问题？

huangapple
go评论127阅读模式
英文:

Go: How to deal with Memory leaks while returning a CString?

问题

我有以下的函数签名，然后返回一个JSON字符串：

  1. func getData(symbol, day, month, year *C.char) *C.char {
  2.   combine, _ := json.Marshal(combineRecords)
  3.   log.Println(string(combine))
  4.   return C.CString(string(combine))
  5. }

然后在Python中调用Go代码：

  1. import ctypes
  2. from time import sleep
  3. library = ctypes.cdll.LoadLibrary('./deribit.so')
  4. get_data = library.getData
  6. # 让Python将其值转换为C表示形式。
  7. # get_data.argtypes = [ctypes.c_char_p, ctypes.c_char_p,ctypes.c_char_p,ctypes.c_char_p]
  8. get_data.restype = ctypes.c_char_p
  10. for i in range(1,100):
  11.     j= get_data("BTC".encode("utf-8"), "5".encode("utf-8"), "JAN".encode("utf-8"), "23".encode("utf-8"))
  12.     # j= get_data(b"BTC", b"3", b"JAN", b"23")
  13.     print('prnting in Python')
  14.     # print(j)
  15.     sleep(1)

在Python端，它按预期工作得很好，但我担心当函数在Python端的循环中被调用时会出现内存泄漏。

我该如何处理内存泄漏？我应该返回bytes而不是CString，并在Python端处理字节以避免内存泄漏吗？我在这个链接中找到了处理内存泄漏的方法，但不知道编组后返回的JSON字符串的大小。

英文:

I have the following function signature which then return a JSON string

  1. func getData(symbol, day, month, year *C.char) *C.char {
  2.   combine, _ := json.Marshal(combineRecords)
  3.   log.Println(string(combine))
  4.   return C.CString(string(combine))
  5. }

The Go code is then being called in Python

  1. import ctypes
  2. from time import sleep
  3. library = ctypes.cdll.LoadLibrary('./deribit.so')
  4. get_data = library.getData
  6. # Make python convert its values to C representation.
  7. # get_data.argtypes = [ctypes.c_char_p, ctypes.c_char_p,ctypes.c_char_p,ctypes.c_char_p]
  8. get_data.restype = ctypes.c_char_p
  10. for i in range(1,100):
  11.     j= get_data("BTC".encode("utf-8"), "5".encode("utf-8"), "JAN".encode("utf-8"), "23".encode("utf-8"))
  12.     # j= get_data(b"BTC", b"3", b"JAN", b"23")
  13.     print('prnting in Python')
  14.     # print(j)
  15.     sleep(1)

It works fine as expected on the Python side but I fear memory leaks when the function will be called in a loop at the Python end.

How do I deal with memory leaks? should I return bytes instead of a CString and deal bytes at Python end to avoid memory leaks? I did find this link to deal with it but somehow I do not know the size of JSON string returned after marshalling

答案1

得分: 1

你是对的，你必须使用C.free来释放它。

https://pkg.go.dev/cmd/cgo

// Go字符串转换为C字符串
// 使用malloc在C堆上分配C字符串。
// 调用者有责任安排它被释放，例如通过调用C.free（如果需要，请确保包含stdlib.h）。
func C.CString(string) *C.char

英文:

You are right, you have to free it by using C.free

https://pkg.go.dev/cmd/cgo

  1. // Go string to C string
  2. // The C string is allocated in the C heap using malloc.
  3. // It is the caller's responsibility to arrange for it to be
  4. // freed, such as by calling C.free (be sure to include stdlib.h
  5. // if C.free is needed).
  6. func C.CString(string) *C.char

答案2

得分: 0

Python代码应该如下所示：

  1. import ctypes
  2. from time import sleep
  4. library = ctypes.CDLL('./stackoverflow.so')
  5. get_data = library.GetData
  6. free_me = library.FreeMe
  7. free_me.argtypes = [ctypes.POINTER(ctypes.c_char)]
  8. get_data.restype = ctypes.POINTER(ctypes.c_char)
  10. for i in range(1, 100):
  11.     j = get_data("", "", "")
  12.     print(ctypes.c_char_p.from_buffer(j).value)
  13.     free_me(j)
  14.     sleep(1)

Go代码应该如下所示：

  1. package main
  3. /*
  4. #include <stdlib.h>
  5. */
  6. import "C"
  7. import (
  8. 	"log"
  9. 	"unsafe"
  10. )
  12. //export GetData
  13. func GetData(symbol, day, month, year *C.char) *C.char {
  14. 	combine := "combine"
  15. 	log.Println(string(combine))
  16. 	return C.CString(string(combine))
  17. }
  19. //export FreeMe
  20. func FreeMe(data *C.char) {
  21. 	C.free(unsafe.Pointer(data))
  22. }
  24. func main() {}

使用以下命令行生成共享库：

  1. python3 --version
  2. Python 3.8.10
  3. go version
  4. go version go1.19.2 linux/amd64
  5. go build -o stackoverflow.so -buildmode=c-shared github.com/sjeandeaux/stackoverflow
  6. python3 stackoverflow.py
  7. 2023/01/03 13:54:14 combine
  8. b'combine'
  9. ...

Dockerfile内容如下：

  1. FROM ubuntu:18.04
  3. RUN apt-get update -y && apt-get install python -y
  5. COPY stackoverflow.so stackoverflow.so
  6. COPY stackoverflow.py stackoverflow.py
  8. CMD ["python", "stackoverflow.py"]

使用以下命令构建Docker镜像：

  1. docker build --tag stackoverflow .
  2. docker run -ti stackoverflow
  3. 2023/01/03 15:04:24 combine
  4. b'combine'
  5. ...
英文:

The python should look like:

  1. import ctypes
  2. from time import sleep
  3. library = ctypes.CDLL(&#39;./stackoverflow.so&#39;)
  4. get_data = library.GetData
  5. free_me = library.FreeMe
  6. free_me.argtypes = [ctypes.POINTER(ctypes.c_char)]
  7. get_data.restype = ctypes.POINTER(ctypes.c_char)
  9. for i in range(1,100):
  10.   j = get_data(&quot;&quot;, &quot;&quot;, &quot;&quot;)
  11.   print(ctypes.c_char_p.from_buffer(j).value)
  12.   free_me(j)
  13.   sleep(1)

The go should look like:

  1. package main
  2. /*
  3. #include &lt;stdlib.h&gt;
  4. */
  5. import &quot;C&quot;
  6. import (
  7.   &quot;log&quot;
  8.   &quot;unsafe&quot;
  9. )
  11. //export GetData
  12. func GetData(symbol, day, month, year *C.char) *C.char {
  13.   combine := &quot;combine&quot;
  14.   log.Println(string(combine))
  15.   return C.CString(string(combine))
  16. }
  18. //export FreeMe
  19. func FreeMe(data *C.char) {
  20.   C.free(unsafe.Pointer(data))
  21. }
  23. func main() {}

And use this command line to generate the shared library:

  1. python3 --version
  2. Python 3.8.10 
  3. go version
  4. go version go1.19.2 linux/amd64
  5. go build -o stackoverflow.so -buildmode=c-shared github.com/sjeandeaux/stackoverflow
  6. python3 stackoverflow.py 
  7. 2023/01/03 13:54:14 combine
  8. b&#39;combine&#39;                                                                                                                                  
  9. ...
  1. FROM ubuntu:18.04
  3. RUN apt-get update -y &amp;&amp; apt-get install python -y
  5. COPY stackoverflow.so stackoverflow.so
  6. COPY stackoverflow.py stackoverflow.py
  8. CMD [&quot;python&quot;, &quot;stackoverflow.py&quot;]
  1. docker build --tag stackoverflow .
  2. docker run -ti stackoverflow
  3. 2023/01/03 15:04:24 combine
  4. b&#39;combine&#39;
  5. ...

huangapple
  • 本文由 发表于 2023年1月3日 16:10:28
  • 转载请务必保留本文链接：https://go.coder-hub.com/74990578.html
  • cgo
  • ctypes
  • go
  • memory-leaks
  • python
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定