英文:
Will linux kernel 5 or above affect the memory release of go1.15?
问题
你好！以下是翻译好的内容：
你使用的Go版本是什么（go version）？
go版本1.15
这个问题在最新版本中是否重现？
没有
你使用的操作系统和处理器架构是什么（go env）？
Linux 5.4.54-1.0.0.std7c.el7.2.x86_64 #1 SMP Mon Nov 16 10:02:20 CST 2020 x86_64 x86_64 x86_64 GNU/Linux
你看到了什么？
下面是一个异常机器的屏幕截图
你做了什么？
内存使用量太高，但通过分析pprof的堆文件，没有发现内存泄漏。
得出这个结论的原因是：
-
我的服务部署在k8s上，还有其他相同的服务。pprof文件的分析结果与这个有问题的服务类似（Linux内核版本为4.9.2-3.0.0.std7b.el7.5.x86_64 cmd/cgo：使用gcc 4.4.1失败 #1 SMP Thu Apr 26 17:33:02 CST 2018 x86_64 x86_64 x86_64 GNU/Linux）。
-
我参考了下面链接中的MADV_FREE问题。通过分析/proc/1/smaps文件中的LazyFree，我发现它全部为0，并且所有机器的Linux内核版本都是4.5或以上。从逻辑上讲，应该会出现相同的问题，但实际上没有。runtime：在Linux上默认使用MADV_DONTNEED #42330
gotrace记录正常
你期望看到什么？
为什么这个服务的内存使用量很高？Linux内核版本5及以上对golang内存管理有什么影响？
英文:
What version of Go are you using (go version)?
go version 1.15
Does this issue reproduce with the latest release?
No
What operating system and processor architecture are you using (go env)?
Linux 5.4.54-1.0.0.std7c.el7.2.x86_64 #1 SMP Mon Nov 16 10:02:20 CST 2020 x86_64 x86_64 x86_64 GNU/Linux
What did you see instead?
Below is a screenshot of the abnormal machine
What did you do?
The memory usage is too high, but there is no memory leak by analyzing the heap file of pprof.
The reason for this conclusion is:
-
My service is deployed on k8s, and there are other same services. The pprof file analysis results are similar to this problematic one (linux kernel version is 4.9.2-3.0.0.std7b.el7.5.x86_64 cmd/cgo: fails with gcc 4.4.1 #1 SMP Thu Apr 26 17:33:02 CST 2018 x86_64 x86_64 x86_64 GNU/Linux).
-
I refer to the MADV_FREE problem in the link below. By analyzing the LazyFree of the /proc/1/smaps file, I found that it is all 0, and all machines are linux kernel 4.5 or above. Logically, the same problem will occur, but it does not No. runtime: default to MADV_DONTNEED on Linux #42330
gotrace records are normal
What did you expect to see?
Why is the memory usage of this service high? What impact does Linux kernel version 5 and above have on golang memory management?
答案1
得分: 3
这与内核版本无关。
Go 1.15 默认使用 MADV_FREE 来释放内存。内核会非常懒散地释放这部分内存，导致 RSS（Resident Set Size，进程驻留内存大小）增加，直到内存实际上需要在其他地方使用时才会释放。因此，RSS 并不能准确反映 Go 程序的实际内存使用情况。
Go 1.16+ 默认使用 MADV_DONTNEED，这会导致内核更及时地释放内存。
详细信息请参考 https://go.dev/doc/go1.16#runtime。
在 Go 1.15 上，你可以使用 GODEBUG=madvdontneed=1 环境变量，或者最好升级到支持的 Go 版本（例如 Go 1.19）。Go 1.20 即将发布，也是一个不错的目标。
英文:
It's not related to the kernel version.
Go 1.15 used MADV_FREE by default to release memory. This memory was released very lazily by the kernel resulting in larger RSS until the memory was actually needed elsewhere. Hence RSS didn't reflect anything close to the Go programs actual usage.
Go 1.16+ defaults to using MADV_DONTNEED, which results in memory being released much more promptly by the kernel.
See https://go.dev/doc/go1.16#runtime for more details.
You can use the GODEBUG=madvdontneed=1 environment variable on Go 1.15, or preferably upgrade to a supported version of Go (eg, Go 1.19). Go 1.20 will be released soon, so that would be a good target too.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论