如何处理返回 CString 时的内存泄漏问题?

huangapple go评论98阅读模式
英文:

Go: How to deal with Memory leaks while returning a CString?

问题

我有以下的函数签名,然后返回一个JSON字符串:

func getData(symbol, day, month, year *C.char) *C.char {
  combine, _ := json.Marshal(combineRecords)
  log.Println(string(combine))
  return C.CString(string(combine))
}

然后在Python中调用Go代码:

import ctypes
from time import sleep
library = ctypes.cdll.LoadLibrary('./deribit.so')
get_data = library.getData

# 让Python将其值转换为C表示形式。
# get_data.argtypes = [ctypes.c_char_p, ctypes.c_char_p,ctypes.c_char_p,ctypes.c_char_p]
get_data.restype = ctypes.c_char_p

for i in range(1,100):
    j= get_data("BTC".encode("utf-8"), "5".encode("utf-8"), "JAN".encode("utf-8"), "23".encode("utf-8"))
    # j= get_data(b"BTC", b"3", b"JAN", b"23")
    print('prnting in Python')
    # print(j)
    sleep(1)

在Python端,它按预期工作得很好,但我担心当函数在Python端的循环中被调用时会出现内存泄漏。

我该如何处理内存泄漏?我应该返回bytes而不是CString,并在Python端处理字节以避免内存泄漏吗?我在这个链接中找到了处理内存泄漏的方法,但不知道编组后返回的JSON字符串的大小。

英文:

I have the following function signature which then return a JSON string

func getData(symbol, day, month, year *C.char) *C.char {
  combine, _ := json.Marshal(combineRecords)
  log.Println(string(combine))
  return C.CString(string(combine))
}

The Go code is then being called in Python

import ctypes
from time import sleep
library = ctypes.cdll.LoadLibrary('./deribit.so')
get_data = library.getData

# Make python convert its values to C representation.
# get_data.argtypes = [ctypes.c_char_p, ctypes.c_char_p,ctypes.c_char_p,ctypes.c_char_p]
get_data.restype = ctypes.c_char_p

for i in range(1,100):
    j= get_data("BTC".encode("utf-8"), "5".encode("utf-8"), "JAN".encode("utf-8"), "23".encode("utf-8"))
    # j= get_data(b"BTC", b"3", b"JAN", b"23")
    print('prnting in Python')
    # print(j)
    sleep(1)

It works fine as expected on the Python side but I fear memory leaks when the function will be called in a loop at the Python end.

How do I deal with memory leaks? should I return bytes instead of a CString and deal bytes at Python end to avoid memory leaks? I did find this link to deal with it but somehow I do not know the size of JSON string returned after marshalling

答案1

得分: 1

你是对的,你必须使用C.free来释放它。

https://pkg.go.dev/cmd/cgo

// Go字符串转换为C字符串
// 使用malloc在C堆上分配C字符串。
// 调用者有责任安排它被释放,例如通过调用C.free(如果需要,请确保包含stdlib.h)。
func C.CString(string) *C.char

英文:

You are right, you have to free it by using C.free

https://pkg.go.dev/cmd/cgo

// Go string to C string
// The C string is allocated in the C heap using malloc.
// It is the caller's responsibility to arrange for it to be
// freed, such as by calling C.free (be sure to include stdlib.h
// if C.free is needed).
func C.CString(string) *C.char

答案2

得分: 0

Python代码应该如下所示:

import ctypes
from time import sleep

library = ctypes.CDLL('./stackoverflow.so')
get_data = library.GetData
free_me = library.FreeMe
free_me.argtypes = [ctypes.POINTER(ctypes.c_char)]
get_data.restype = ctypes.POINTER(ctypes.c_char)

for i in range(1, 100):
    j = get_data("", "", "")
    print(ctypes.c_char_p.from_buffer(j).value)
    free_me(j)
    sleep(1)

Go代码应该如下所示:

package main

/*
#include <stdlib.h>
*/
import "C"
import (
	"log"
	"unsafe"
)

//export GetData
func GetData(symbol, day, month, year *C.char) *C.char {
	combine := "combine"
	log.Println(string(combine))
	return C.CString(string(combine))
}

//export FreeMe
func FreeMe(data *C.char) {
	C.free(unsafe.Pointer(data))
}

func main() {}

使用以下命令行生成共享库:

python3 --version
Python 3.8.10
go version
go version go1.19.2 linux/amd64
go build -o stackoverflow.so -buildmode=c-shared github.com/sjeandeaux/stackoverflow
python3 stackoverflow.py
2023/01/03 13:54:14 combine
b'combine'
...

Dockerfile内容如下:

FROM ubuntu:18.04

RUN apt-get update -y && apt-get install python -y

COPY stackoverflow.so stackoverflow.so
COPY stackoverflow.py stackoverflow.py

CMD ["python", "stackoverflow.py"]

使用以下命令构建Docker镜像:

docker build --tag stackoverflow .
docker run -ti stackoverflow
2023/01/03 15:04:24 combine
b'combine'
...
英文:

The python should look like:

import ctypes
from time import sleep
library = ctypes.CDLL(&#39;./stackoverflow.so&#39;)
get_data = library.GetData
free_me = library.FreeMe
free_me.argtypes = [ctypes.POINTER(ctypes.c_char)]
get_data.restype = ctypes.POINTER(ctypes.c_char)

for i in range(1,100):
  j = get_data(&quot;&quot;, &quot;&quot;, &quot;&quot;)
  print(ctypes.c_char_p.from_buffer(j).value)
  free_me(j)
  sleep(1)

The go should look like:

package main
/*
#include &lt;stdlib.h&gt;
*/
import &quot;C&quot;
import (
  &quot;log&quot;
  &quot;unsafe&quot;
)

//export GetData
func GetData(symbol, day, month, year *C.char) *C.char {
  combine := &quot;combine&quot;
  log.Println(string(combine))
  return C.CString(string(combine))
}

//export FreeMe
func FreeMe(data *C.char) {
  C.free(unsafe.Pointer(data))
}

func main() {}

And use this command line to generate the shared library:

python3 --version
Python 3.8.10 
go version
go version go1.19.2 linux/amd64
go build -o stackoverflow.so -buildmode=c-shared github.com/sjeandeaux/stackoverflow
python3 stackoverflow.py 
2023/01/03 13:54:14 combine
b&#39;combine&#39;                                                                                                                                  
...
FROM ubuntu:18.04

RUN apt-get update -y &amp;&amp; apt-get install python -y

COPY stackoverflow.so stackoverflow.so
COPY stackoverflow.py stackoverflow.py

CMD [&quot;python&quot;, &quot;stackoverflow.py&quot;]
docker build --tag stackoverflow .
docker run -ti stackoverflow
2023/01/03 15:04:24 combine
b&#39;combine&#39;
...

huangapple
  • 本文由 发表于 2023年1月3日 16:10:28
  • 转载请务必保留本文链接:https://go.coder-hub.com/74990578.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定