英文:
Profiling a Go program spanning several runs
问题
我想要对一个Go程序在不同的操作系统级别设置下的性能进行分析。我知道可以通过$ go test -cpuprofile cpu.prof -memprofile mem.prof -bench .来获取单次运行的性能分析结果。然而，我不知道如何以一种可以进行可视化或编程比较结果的方式来汇总这些信息。
为了在Xonsh脚本语言中呈现一个示意图，它是Python和Bash之间的混合语言。然而，我也可以接受纯Bash编写的建议。
for i in range(n):
change_system_settings()
# 运行 'go test' 并将结果保存在 cpu0.prof, cpu1.prof, cpu2.prof 等文件中
@(f'go test -cpuprofile cpu{i}.prof -memprofile mem{i}.prof -bench .'.split())
该脚本会更改系统设置，并通过性能分析器运行程序n次。现在，在这个过程之后，我可能会得到几十个单独的.prof文件。我希望能够对它们进行整体的查看，比较不同运行之间的内存和CPU使用情况，甚至运行数值测试来确定哪个运行是最优的。
英文:
I want to profile a Go program's performance between different runs with different OS-level settings. I'm aware that I can get profiles for single runs via $ go test -cpuprofile cpu.prof -memprofile mem.prof -bench .. However I don't know how to aggregate the information in such a way that I can compare the results either visually or programmatically.
To present a sketch in Xonsh scripting language, which is a creole between Python and Bash. However I'm happy to accept suggestion written in pure Bash as well.
for i in range(n):
change_system_settings()
# Run 'go test' and save the results in cpu0.prof, cpu1.prof, cpu2.prof etc.
@(f'go test -cpuprofile cpu{i}.prof -memprofile mem{i}.prof -bench .'.split())
The script changes the system settings and runs the program through profiler n times. Now, after the process I'm left with possibly dozens of individual .prof files. I would like to have a holistic view of them, compare the memory and CPU usage between runs and even run numeric tests to see which run was optimal.
答案1
得分: 2
如果你使用Go语言的pprof来对你的Go程序进行性能分析，该库提供了一个Merge方法，可以将多个pprof输出文件合并为一个文件。
该库的地址是github.com/google/pprof，你只需要在Go脚本中导入它：
import "github.com/google/pprof/profile"
然后，你需要将所有的pprof文件加载到一个数组中。假设你已经完成了这一步，将所有的文件（使用os.Open()）加载到一个名为allFiles的数组中，你可以使用以下方法将它们合并：
result, err := profile.Merge(allFiles)
然后，你可以将合并后的数据输出到一个新文件中，使用os.OpenFile(...)打开文件，写入数据，然后关闭文件。
老实说，我现在没有测试过这个方法，但我记得很久以前我们是这样做的。所以在你的测试脚本的for循环结束后，你可以调用这个Go语言脚本。
文档：https://github.com/google/pprof/blob/master/doc/README.md
英文:
If you use GoLang's pprof to profile your Go program, the library has a Merge method that merges multiple pprof output files into one.
The library is github.com/google/pprof, so you just import it in a Go script:
import ('github.com/google/pprof/profile')
Then you'll need to load all your pprof files into one array. If we consider that you did that and you have all your files loaded (using os.Open()) in an array called allFiles, you merge them using the following method:
result, err := profile.Merge(allFiles)
Then you output the merged data into a new file, using os.OpenFile(...), writing to this file, then closing it.
I haven't tested this right now honestly, but I remember this is how we did it a long time ago. So technically, you could invoke this golang script after your for loop is done in your test script.
Documentation: https://github.com/google/pprof/blob/master/doc/README.md
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论