英文:
finding memory leaks in golang program - reflect.Value.call meaning
问题
我正在尝试找到一个内存泄漏的代码片段。
在启动全新的Web应用程序后,它占用6 MB的内存。
经过大约12,000个请求后,它占用28 MB的内存。
我在启动后保存了它的堆内存快照:
curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/6mb.heap
然后在12,000个请求后保存了堆内存快照:
curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/28mb.heap
然后我尝试查看分配的对象差异:
go tool pprof -alloc_objects -base ~/debug/heavyHeap/6mb.heap $GOPATH/myBin ~/debug/heavyHeap/28mb.heap
运行top命令:
进入交互模式(输入"help"获取命令列表)
(pprof) top
总共 83397023 个对象中的 73949086 个(占总数的 88.67%)
删除了 299 个节点(累计 <= 416985)
显示前 10 个节点(累计 >= 1802254)
flat flat% sum% cum cum%
62308988 74.71% 74.71% 62521981 74.97% reflect.Value.call
2413961 2.89% 77.61% 2424884 2.91% calldb.fromToDiff
1769493 2.12% 79.73% 3796564 4.55% gopkg.in/mgo.v2/bson.(*decoder).readElemTo
1622034 1.94% 81.67% 1622034 1.94% gopkg.in/mgo.v2/bson.(*decoder).readCStr
1270739 1.52% 83.20% 1401813 1.68% reflect.(*structType).FieldByNameFunc
1130028 1.35% 84.55% 1130028 1.35% reflect.Value.MapKeys
933704 1.12% 85.67% 933704 1.12% gopkg.in/mgo.v2/bson.(*decoder).readStr
927261 1.11% 86.79% 946166 1.13% fmt.Sprintf
819209 0.98% 87.77% 1119590 1.34% my.AnchorWithClassAndDisabledAndStyle
我列出了最重的项 reflect.Value.call
:
(pprof) list reflect.Value.call
总共: 83397023
ROUTINE ======================== reflect.Value.call in /usr/local/go/src/reflect/value.go
62308988 62521981 (flat, cum) 占总数的 74.97%
. . 366: }
. . 367: }
. . 368: if !isSlice && t.IsVariadic() {
. . 369: // prepare slice for remaining values
. . 370: m := len(in) - n
. 81921 371: slice := MakeSlice(t.In(n), m, m)
. . 372: elem := t.In(n).Elem()
. . 373: for i := 0; i < m; i++ {
. . 374: x := in[n+i]
. . 375: if xt := x.Type(); !xt.AssignableTo(elem) {
. . 376: panic("reflect: cannot use " + xt.String() + " as type " + elem.String() + " in " + op)
. . 377: }
. 131072 378: slice.Index(i).Set(x)
. . 379: }
. . 380: origIn := in
. . 381: in = make([]Value, n+1)
. . 382: copy(in[:n], origIn)
. . 383: in[n] = slice
. . 384: }
. . 385:
. . 386: nin := len(in)
. . 387: if nin != t.NumIn() {
. . 388: panic("reflect.Value.Call: wrong argument count")
. . 389: }
. . 390: nout := t.NumOut()
. . 391:
. . 392: // Compute frame type, allocate a chunk of memory for frame
. . 393: frametype, _, retOffset, _ := funcLayout(t, rcvrtype)
32769 32769 394: args := unsafe_New(frametype)
. . 395: off := uintptr(0)
. . 396:
. . 397: // Copy inputs into args.
. . 398: if rcvrtype != nil {
. . 399: storeRcvr(rcvr, args)
. . 400: off = ptrSize
. . 401: }
. . 402: for i, v := range in {
. . 403: v.mustBeExported()
. . 404: targ := t.In(i).(*rtype)
. . 405: a := uintptr(targ.align)
. . 406: off = (off + a - 1) &^ (a - 1)
. . 407: n := targ.size
. . 408: addr := unsafe.Pointer(uintptr(args) + off)
. . 409: v = v.assignTo("reflect.Value.Call", targ, addr)
. . 410: if v.flag&flagIndir != 0 {
. . 411: memmove(addr, v.ptr, n)
. . 412: } else {
. . 413: *(*unsafe.Pointer)(addr) = v.ptr
. . 414: }
. . 415: off += n
. . 416: }
. . 417:
. . 418: // Call.
62243451 62243451 419: call(fn, args, uint32(frametype.size), uint32(retOffset))
. . 420:
. . 421: // For testing; see TestCallMethodJump.
. . 422: if callGC {
. . 423: runtime.GC()
. . 424: }
. . 425:
. . 426: // Copy return values out of args.
32768 32768 427: ret := make([]Value, nout)
. . 428: off = retOffset
. . 429: for i := 0; i < nout; i++ {
. . 430: tv := t.Out(i)
. . 431: a := uintptr(tv.Align())
. . 432: off = (off + a - 1) &^ (a - 1)
但是所有这些信息都没有给出我代码中的内存泄漏位置的线索。
reflect.Value.call
实际上是什么意思?
我不记得在我的代码中使用过reflect.Value.call
。
英文:
I'm trying to find piece of code which is memory leaking.
After launching fresh web application, it's 6 MB.
After about 12k requests, it's 28 MB.
I saved its heap just after launching
curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/6mb.heap
And after 12k requests:
curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/28mb.heap
Then I am trying to see allocated objects difference:
go tool pprof -alloc_objects -base ~/debug/heavyHeap/6mb.heap $GOPATH/myBin ~/debug/heavyHeap/28mb.heap
Run top command:
Entering interactive mode (type "help" for commands)
(pprof) top
73949086 of 83397023 total (88.67%)
Dropped 299 nodes (cum <= 416985)
Showing top 10 nodes out of 117 (cum >= 1802254)
flat flat% sum% cum cum%
62308988 74.71% 74.71% 62521981 74.97% reflect.Value.call
2413961 2.89% 77.61% 2424884 2.91% calldb.fromToDiff
1769493 2.12% 79.73% 3796564 4.55% gopkg.in/mgo.v2/bson.(*decoder).readElemTo
1622034 1.94% 81.67% 1622034 1.94% gopkg.in/mgo.v2/bson.(*decoder).readCStr
1270739 1.52% 83.20% 1401813 1.68% reflect.(*structType).FieldByNameFunc
1130028 1.35% 84.55% 1130028 1.35% reflect.Value.MapKeys
933704 1.12% 85.67% 933704 1.12% gopkg.in/mgo.v2/bson.(*decoder).readStr
927261 1.11% 86.79% 946166 1.13% fmt.Sprintf
819209 0.98% 87.77% 1119590 1.34% my.AnchorWithClassAndDisabledAndStyle
I list the heaviest item reflect.Value.call
:
(pprof) list reflect.Value.call
Total: 83397023
ROUTINE ======================== reflect.Value.call in /usr/local/go/src/reflect/value.go
62308988 62521981 (flat, cum) 74.97% of Total
. . 366: }
. . 367: }
. . 368: if !isSlice && t.IsVariadic() {
. . 369: // prepare slice for remaining values
. . 370: m := len(in) - n
. 81921 371: slice := MakeSlice(t.In(n), m, m)
. . 372: elem := t.In(n).Elem()
. . 373: for i := 0; i < m; i++ {
. . 374: x := in[n+i]
. . 375: if xt := x.Type(); !xt.AssignableTo(elem) {
. . 376: panic("reflect: cannot use " + xt.String() + " as type " + elem.String() + " in " + op)
. . 377: }
. 131072 378: slice.Index(i).Set(x)
. . 379: }
. . 380: origIn := in
. . 381: in = make([]Value, n+1)
. . 382: copy(in[:n], origIn)
. . 383: in[n] = slice
. . 384: }
. . 385:
. . 386: nin := len(in)
. . 387: if nin != t.NumIn() {
. . 388: panic("reflect.Value.Call: wrong argument count")
. . 389: }
. . 390: nout := t.NumOut()
. . 391:
. . 392: // Compute frame type, allocate a chunk of memory for frame
. . 393: frametype, _, retOffset, _ := funcLayout(t, rcvrtype)
32769 32769 394: args := unsafe_New(frametype)
. . 395: off := uintptr(0)
. . 396:
. . 397: // Copy inputs into args.
. . 398: if rcvrtype != nil {
. . 399: storeRcvr(rcvr, args)
. . 400: off = ptrSize
. . 401: }
. . 402: for i, v := range in {
. . 403: v.mustBeExported()
. . 404: targ := t.In(i).(*rtype)
. . 405: a := uintptr(targ.align)
. . 406: off = (off + a - 1) &^ (a - 1)
. . 407: n := targ.size
. . 408: addr := unsafe.Pointer(uintptr(args) + off)
. . 409: v = v.assignTo("reflect.Value.Call", targ, addr)
. . 410: if v.flag&flagIndir != 0 {
. . 411: memmove(addr, v.ptr, n)
. . 412: } else {
. . 413: *(*unsafe.Pointer)(addr) = v.ptr
. . 414: }
. . 415: off += n
. . 416: }
. . 417:
. . 418: // Call.
62243451 62243451 419: call(fn, args, uint32(frametype.size), uint32(retOffset))
. . 420:
. . 421: // For testing; see TestCallMethodJump.
. . 422: if callGC {
. . 423: runtime.GC()
. . 424: }
. . 425:
. . 426: // Copy return values out of args.
32768 32768 427: ret := make([]Value, nout)
. . 428: off = retOffset
. . 429: for i := 0; i < nout; i++ {
. . 430: tv := t.Out(i)
. . 431: a := uintptr(tv.Align())
. . 432: off = (off + a - 1) &^ (a - 1)
But all this stuff don't give a clue where are memory leaks in my code.
What does reflect.Value.call
actually mean?
I don't remember that I used reflect.Value.call
in my code.
答案1
得分: 4
包gopkg.in/mgo.v2/bson
使用反射来进行BSON的(解)编组。但是由于问题11786,实际导致内存泄漏的方法在性能分析中没有显示出来。
解决方法是向go tool pprof
传递标志-runtime
。
英文:
Package gopkg.in/mgo.v2/bson
uses reflection to (un)marshal BSON. But the actual method leaking memory is not shown in the profile because of issue 11786.
The workaround is to pass the flag -runtime
to go tool pprof
.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论