英文:
profile kubectl using pprof
问题
在kubernetes源代码中,有一段处理性能分析的代码块,但我无法访问这些端点:
在kubernetes/pkgs/kubelet/server/stats/server.go中
func (s *Server) InstallProfilingHandler(enableProfilingLogHandler bool, enableContentionProfiling bool) {
s.addMetricsBucketMatcher("debug")
if !enableProfilingLogHandler {
s.restfulCont.Handle(pprofBasePath, getHandlerForDisabledEndpoint("profiling endpoint is disabled."))
return
}
handlePprofEndpoint := func(req *restful.Request, resp *restful.Response) {
name := strings.TrimPrefix(req.Request.URL.Path, pprofBasePath)
switch name {
case "profile":
pprof.Profile(resp, req.Request)
case "symbol":
pprof.Symbol(resp, req.Request)
case "cmdline":
pprof.Cmdline(resp, req.Request)
case "trace":
pprof.Trace(resp, req.Request)
default:
pprof.Index(resp, req.Request)
}
}
// 设置pprof处理程序。
ws := new(restful.WebService).Path(pprofBasePath)
ws.Route(ws.GET("/{subpath:*}").To(handlePprofEndpoint)).Doc("pprof endpoint")
s.restfulCont.Add(ws)
if enableContentionProfiling {
goruntime.SetBlockProfileRate(1)
}
}
我不知道pprof工具使用的端口,但我通过以下命令找到了它:
controller-0:/home/sysadmin/go/bin# netstat -atlpn | grep kubelet
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 184856/kubelet
tcp 0 0 192.168.206.2:49720 192.168.206.1:6443 ESTABLISHED 184856/kubelet
tcp6 0 0 :::10250 :::* LISTEN 184856/kubelet
然后我尝试了以下命令:
controller-0:/home/sysadmin/go/bin# ./go tool pprof http://localhost:6443/debug/pprof/mutex
Fetching profile over HTTP from http://localhost:6443/debug/pprof/mutex
http://localhost:6443/debug/pprof/mutex: server response: 400 Bad Request
failed to fetch any source profiles
有没有人知道我应该尝试什么来访问pprof端点?或者如何尝试其他方法来对kubelet进程进行性能分析?
英文:
In the kubernetes source code there is a block of code that handles the profiling part but I can not acces the endpoints:
in kubernetes/pkgs/kubelet/server/stats/server.go
func (s *Server) InstallProfilingHandler(enableProfilingLogHandler bool, enableContentionProfiling bool) {
s.addMetricsBucketMatcher("debug")
if !enableProfilingLogHandler {
s.restfulCont.Handle(pprofBasePath, getHandlerForDisabledEndpoint("profiling endpoint is disabled."))
return
}
handlePprofEndpoint := func(req *restful.Request, resp *restful.Response) {
name := strings.TrimPrefix(req.Request.URL.Path, pprofBasePath)
switch name {
case "profile":
pprof.Profile(resp, req.Request)
case "symbol":
pprof.Symbol(resp, req.Request)
case "cmdline":
pprof.Cmdline(resp, req.Request)
case "trace":
pprof.Trace(resp, req.Request)
default:
pprof.Index(resp, req.Request)
}
}
// Setup pprof handlers.
ws := new(restful.WebService).Path(pprofBasePath)
ws.Route(ws.GET("/{subpath:*}").To(handlePprofEndpoint)).Doc("pprof endpoint")
s.restfulCont.Add(ws)
if enableContentionProfiling {
goruntime.SetBlockProfileRate(1)
}
}
I do not know the port used by the pprof tool but I found it using:
controller-0:/home/sysadmin/go/bin# netstat -atlpn | grep kubelet
tcp 0 0 127.0.0.1:10248 0.0.0.0:* LISTEN 184856/kubelet
tcp 0 0 192.168.206.2:49720 192.168.206.1:6443 ESTABLISHED 184856/kubelet
tcp6 0 0 :::10250 :::* LISTEN 184856/kubelet
Then I have tried
controller-0:/home/sysadmin/go/bin# ./go tool pprof http://localhost:6443/debug/pprof/mutex
Fetching profile over HTTP from http://localhost:6443/debug/pprof/mutex
http://localhost:6443/debug/pprof/mutex: server response: 400 Bad Request
failed to fetch any source profiles
Is there anyone that knows what should I try to access the pprof endpoints? or how to try a different approach to profile the kubelet process?
答案1
得分: 3
尝试一下:
$ kubectl proxy &
开始在 127.0.0.1:8001 上提供服务
$ go tool pprof "http://localhost:8001/api/v1/nodes/${NODE}/proxy/debug/pprof/profile"
当你启动 kubectl proxy
后,所有对 http://localhost:8001/api/v1/nodes/${NODE}/proxy/
的请求都会发送到运行在 ${NODE}
上的 kubelet。你可以添加任何你想要的路径。比如 /debug/pprof/profile
或者 debug/pprof/heap
,或者其他任何你想要的路径。
英文:
Try:
$ kubectl proxy &
Starting to serve on 127.0.0.1:8001
$ go tool pprof "http://localhost:8001/api/v1/nodes/${NODE}/proxy/debug/pprof/profile"
When you start kubectl proxy
, all requests to http://localhost:8001/api/v1/nodes/${NODE}/proxy/
go to the kubelet running on the ${NODE}
. You can add any path you want and it. Let it be /debug/pprof/profile
or debug/pprof/heap
or whatever you want.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论