英文:
why does http post request gives me high memory usage in go?
问题
我有一个在K8S容器中运行的go应用程序。它作为一个REST API工作，接收请求并将请求写入Elasticsearch。
我有以下代码：
var r = gin.Default()
r.POST("/logs", func(c *gin.Context) {
fmt.Println("receive log event")
PrintMemUsage()
jsonData, err := ioutil.ReadAll(c.Request.Body)
d := strings.NewReader(jsonData)
http.Post(fmt.Sprintf("%s/_bulk", getEsHost()), "application/json", d)
...
})
在上面的代码中，它监听路径/logs并调用http将数据保存在Elasticsearch中。当我使用下面的函数打印内存使用情况时，我可以看到Alloc持续增加，直到内存用尽。如果我删除http.Post调用，内存使用量始终保持在1到3MB之间。是什么原因导致内存使用量不断增加呢？
func bToMb(b uint64) uint64 {
return b / 1024 / 1024
}
func PrintMemUsage() {
var m runtime.MemStats
runtime.ReadMemStats(&m)
// 有关每个字段的信息，请参阅：https://golang.org/pkg/runtime/#MemStats
fmt.Printf("Alloc = %v MiB", bToMb(m.Alloc))
fmt.Printf("\tTotalAlloc = %v MiB", bToMb(m.TotalAlloc))
fmt.Printf("\tSys = %v MiB", bToMb(m.Sys))
fmt.Printf("\tNumGC = %v\n", m.NumGC)
}
英文:
I have a go application running inside K8S container. It works as a rest api and receive request and write them request to Elasticsearch.
The code I have is:
var r = gin.Default()
r.POST("/logs", func(c *gin.Context) {
fmt.Println("receive log event")
PrintMemUsage()
jsonData, err := ioutil.ReadAll(c.Request.Body)
d := strings.NewReader(jsonData)
http.Post(fmt.Sprintf("%s/_bulk", getEsHost()), "application/json", d)
...
})
}
In above code, it listens on path /logs and call http to save the data in Elasticsearch. When I use below function to print the memory usage, I can see the Alloc keeps increasing until run out of memory. The memory usage is consistently on 1 to 3MB if I remove the http.Post call. What could be the reason causing memory usage keep increasing?
func bToMb(b uint64) uint64 {
return b / 1024 / 1024
}
func PrintMemUsage() {
var m runtime.MemStats
runtime.ReadMemStats(&m)
// For info on each, see: https://golang.org/pkg/runtime/#MemStats
fmt.Printf("Alloc = %v MiB", bToMb(m.Alloc))
fmt.Printf("\tTotalAlloc = %v MiB", bToMb(m.TotalAlloc))
fmt.Printf("\tSys = %v MiB", bToMb(m.Sys))
fmt.Printf("\tNumGC = %v\n", m.NumGC)
}
答案1
得分: 0
http docs中多次提到：
> 客户端在使用完响应体后必须关闭它：
以下是文档中的示例代码：
resp, err := http.Get("http://example.com/")
if err != nil {
// 处理错误
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
// ...
如果你不这样做，就会造成内存泄漏，因为响应体将永远留在内存中。
英文:
The http docs mention a few times that:
> The client must close the response body when finished with it:
Here is the example from the docs:
resp, err := http.Get("http://example.com/")
if err != nil {
// handle error
}
defer resp.Body.Close()
body, err := io.ReadAll(resp.Body)
// ...
If you don't do that, you have a leak, as the body will stay in memory forever.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论