英文:
discrepancy between htop and golang readmemstats
问题
我的程序在启动时加载了大量的数据，然后调用debug.FreeOSMemory()函数，以便立即释放任何多余的空间。
loadDataIntoMem()
debug.FreeOSMemory()
加载到内存后，htop显示该进程的以下信息：
VIRT RES SHR
11.6G 7629M 8000
但是调用runtime.ReadMemStats函数后，显示以下信息：
Alloc 5593336608 5.3G
BuckHashSys 1574016 1.6M
HeapAlloc 5593336610 5.3G
HeapIdle 2607980544 2.5G
HeapInuse 7062446080 6.6G
HeapReleased 2607980544 2.5G
HeapSys 9670426624 9.1G
MCacheInuse 9600 9.4K
MCacheSys 16384 16K
MSpanInuse 106776176 102M
MSpanSys 115785728 111M
OtherSys 25638523 25M
StackInuse 589824 576K
StackSys 589824 576K
Sys 10426738360 9.8G
TotalAlloc 50754542056 48G
- Alloc是从系统获取但尚未释放的内存量（这是常驻内存吗？），但两者之间存在很大的差异。
- 我依赖HeapIdle来判断是否重启程序，即如果HeapIdle超过2GB，则重启程序。在这种情况下，它是2.5GB，即使过了一段时间也没有减少。Golang在将来分配更多内存时应该使用堆空闲，从而减少堆空闲，对吗？
- 如果假设1是错误的，哪个统计数据可以准确告诉我htop中的RES值是什么？
- 我该如何减少HeapIdle的值？
这是在Go 1.4.2、1.5.2和1.6.beta1上尝试的。
英文:
My program loads a lot of data at start up and then calls debug.FreeOSMemory() so that any extra space is given back immediately.
loadDataIntoMem()
debug.FreeOSMemory()
after loading into memory , htop shows me the following for the process
VIRT RES SHR
11.6G 7629M 8000
But a call to runtime.ReadMemStats shows me the following
Alloc 5593336608 5.3G
BuckHashSys 1574016 1.6M
HeapAlloc 5593336610 5.3G
HeapIdle 2607980544 2.5G
HeapInuse 7062446080 6.6G
HeapReleased 2607980544 2.5G
HeapSys 9670426624 9.1G
MCacheInuse 9600 9.4K
MCacheSys 16384 16K
MSpanInuse 106776176 102M
MSpanSys 115785728 111M
OtherSys 25638523 25M
StackInuse 589824 576K
StackSys 589824 576K
Sys 10426738360 9.8G
TotalAlloc 50754542056 48G
- Alloc is the amount obtained from system and not yet freed ( This is
resident memory right ?) But there is a big difference between the two. - I rely on HeapIdle to kill my program i.e if HeapIdle is more than 2 GB, restart - in this case it is 2.5, and isn't going down even after a while. Golang should use from heap idle when allocating more in the future, thus reducing heap idle right ?
- If assumption 1 is wrong, which stat can accurately tell me what the RES value in htop is.
- What can I do to reduce the value of HeapIdle ?
This was tried on go 1.4.2, 1.5.2 and 1.6.beta1
答案1
得分: 1
你的程序的有效内存消耗将是Sys-HeapReleased。这仍然不会完全与操作系统报告的相同，因为操作系统可以根据程序的请求自行选择分配内存。
如果你的程序运行了相当长的时间，多余的内存将会被归还给操作系统，所以不需要调用debug.FreeOSMemory()。垃圾回收器的任务也不是尽可能保持内存使用最低，而是尽可能高效地使用内存。这需要一些开销，并为未来的分配留出空间。
如果你在处理内存使用方面遇到问题，更有成效的做法是对你的程序进行性能分析，看看为什么你的分配量超出了预期，而不是基于对内存的错误假设来终止进程。
英文:
The effective memory consumption of your program will be Sys-HeapReleased. This still won't be exactly what the OS reports, because the OS can choose to allocate memory how it sees fit based on the requests of the program.
If your program runs for any appreciable amount of time, the excess memory will be offered back to the OS so there's no need to call debug.FreeOSMemory(). It's also not the job of the garbage collector to keep memory as low as possible; the goal is to use memory as efficiently as possible. This requires some overhead, and room for future allocations.
If you're having trouble with memory usage, it would be a lot more productive to profile your program and see why you're allocating more than expected, instead of killing your process based on incorrect assumptions about memory.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论