在Go语言程序中查找内存泄漏 – reflect.Value.call的含义

huangapple go评论86阅读模式
英文:

finding memory leaks in golang program - reflect.Value.call meaning

问题

我正在尝试找到一个内存泄漏的代码片段。

在启动全新的Web应用程序后,它占用6 MB的内存。
经过大约12,000个请求后,它占用28 MB的内存。

我在启动后保存了它的堆内存快照:

curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/6mb.heap

然后在12,000个请求后保存了堆内存快照:

curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/28mb.heap

然后我尝试查看分配的对象差异:

go tool pprof -alloc_objects -base ~/debug/heavyHeap/6mb.heap $GOPATH/myBin ~/debug/heavyHeap/28mb.heap

运行top命令:

进入交互模式(输入"help"获取命令列表)
(pprof) top
总共 83397023 个对象中的 73949086 个(占总数的 88.67%)
删除了 299 个节点(累计 <= 416985)
显示前 10 个节点(累计 >= 1802254)
      flat  flat%   sum%        cum   cum%
  62308988 74.71% 74.71%   62521981 74.97%  reflect.Value.call
   2413961  2.89% 77.61%    2424884  2.91%  calldb.fromToDiff
   1769493  2.12% 79.73%    3796564  4.55%  gopkg.in/mgo.v2/bson.(*decoder).readElemTo
   1622034  1.94% 81.67%    1622034  1.94%  gopkg.in/mgo.v2/bson.(*decoder).readCStr
   1270739  1.52% 83.20%    1401813  1.68%  reflect.(*structType).FieldByNameFunc
   1130028  1.35% 84.55%    1130028  1.35%  reflect.Value.MapKeys
    933704  1.12% 85.67%     933704  1.12%  gopkg.in/mgo.v2/bson.(*decoder).readStr
    927261  1.11% 86.79%     946166  1.13%  fmt.Sprintf
    819209  0.98% 87.77%    1119590  1.34%  my.AnchorWithClassAndDisabledAndStyle

我列出了最重的项 reflect.Value.call

(pprof) list reflect.Value.call
总共: 83397023
ROUTINE ======================== reflect.Value.call in /usr/local/go/src/reflect/value.go
  62308988   62521981 (flat, cum) 占总数的 74.97%
         .          .    366:   }
         .          .    367: }
         .          .    368: if !isSlice && t.IsVariadic() {
         .          .    369:   // prepare slice for remaining values
         .          .    370:   m := len(in) - n
         .      81921    371:   slice := MakeSlice(t.In(n), m, m)
         .          .    372:   elem := t.In(n).Elem()
         .          .    373:   for i := 0; i < m; i++ {
         .          .    374:     x := in[n+i]
         .          .    375:     if xt := x.Type(); !xt.AssignableTo(elem) {
         .          .    376:       panic("reflect: cannot use " + xt.String() + " as type " + elem.String() + " in " + op)
         .          .    377:     }
         .     131072    378:     slice.Index(i).Set(x)
         .          .    379:   }
         .          .    380:   origIn := in
         .          .    381:   in = make([]Value, n+1)
         .          .    382:   copy(in[:n], origIn)
         .          .    383:   in[n] = slice
         .          .    384: }
         .          .    385:
         .          .    386: nin := len(in)
         .          .    387: if nin != t.NumIn() {
         .          .    388:   panic("reflect.Value.Call: wrong argument count")
         .          .    389: }
         .          .    390: nout := t.NumOut()
         .          .    391:
         .          .    392: // Compute frame type, allocate a chunk of memory for frame
         .          .    393: frametype, _, retOffset, _ := funcLayout(t, rcvrtype)
     32769      32769    394: args := unsafe_New(frametype)
         .          .    395: off := uintptr(0)
         .          .    396:
         .          .    397: // Copy inputs into args.
         .          .    398: if rcvrtype != nil {
         .          .    399:   storeRcvr(rcvr, args)
         .          .    400:   off = ptrSize
         .          .    401: }
         .          .    402: for i, v := range in {
         .          .    403:   v.mustBeExported()
         .          .    404:   targ := t.In(i).(*rtype)
         .          .    405:   a := uintptr(targ.align)
         .          .    406:   off = (off + a - 1) &^ (a - 1)
         .          .    407:   n := targ.size
         .          .    408:   addr := unsafe.Pointer(uintptr(args) + off)
         .          .    409:   v = v.assignTo("reflect.Value.Call", targ, addr)
         .          .    410:   if v.flag&flagIndir != 0 {
         .          .    411:     memmove(addr, v.ptr, n)
         .          .    412:   } else {
         .          .    413:     *(*unsafe.Pointer)(addr) = v.ptr
         .          .    414:   }
         .          .    415:   off += n
         .          .    416: }
         .          .    417:
         .          .    418: // Call.
  62243451   62243451    419: call(fn, args, uint32(frametype.size), uint32(retOffset))
         .          .    420:
         .          .    421: // For testing; see TestCallMethodJump.
         .          .    422: if callGC {
         .          .    423:   runtime.GC()
         .          .    424: }
         .          .    425:
         .          .    426: // Copy return values out of args.
     32768      32768    427: ret := make([]Value, nout)
         .          .    428: off = retOffset
         .          .    429: for i := 0; i < nout; i++ {
         .          .    430:   tv := t.Out(i)
         .          .    431:   a := uintptr(tv.Align())
         .          .    432:   off = (off + a - 1) &^ (a - 1)

但是所有这些信息都没有给出我代码中的内存泄漏位置的线索。

reflect.Value.call实际上是什么意思?
我不记得在我的代码中使用过reflect.Value.call

英文:

I'm trying to find piece of code which is memory leaking.

After launching fresh web application, it's 6 MB.
After about 12k requests, it's 28 MB.

I saved its heap just after launching

curl -s localhost:6060/debug/pprof/heap &gt; ~/debug/heavyHeap/6mb.heap

And after 12k requests:

curl -s localhost:6060/debug/pprof/heap &gt; ~/debug/heavyHeap/28mb.heap

Then I am trying to see allocated objects difference:

go tool pprof -alloc_objects -base ~/debug/heavyHeap/6mb.heap $GOPATH/myBin ~/debug/heavyHeap/28mb.heap

Run top command:

Entering interactive mode (type &quot;help&quot; for commands)
(pprof) top
73949086 of 83397023 total (88.67%)
Dropped 299 nodes (cum &lt;= 416985)
Showing top 10 nodes out of 117 (cum &gt;= 1802254)
      flat  flat%   sum%        cum   cum%
  62308988 74.71% 74.71%   62521981 74.97%  reflect.Value.call
   2413961  2.89% 77.61%    2424884  2.91%  calldb.fromToDiff
   1769493  2.12% 79.73%    3796564  4.55%  gopkg.in/mgo.v2/bson.(*decoder).readElemTo
   1622034  1.94% 81.67%    1622034  1.94%  gopkg.in/mgo.v2/bson.(*decoder).readCStr
   1270739  1.52% 83.20%    1401813  1.68%  reflect.(*structType).FieldByNameFunc
   1130028  1.35% 84.55%    1130028  1.35%  reflect.Value.MapKeys
    933704  1.12% 85.67%     933704  1.12%  gopkg.in/mgo.v2/bson.(*decoder).readStr
    927261  1.11% 86.79%     946166  1.13%  fmt.Sprintf
    819209  0.98% 87.77%    1119590  1.34%  my.AnchorWithClassAndDisabledAndStyle

I list the heaviest item reflect.Value.call:

(pprof) list reflect.Value.call
Total: 83397023
ROUTINE ======================== reflect.Value.call in /usr/local/go/src/reflect/value.go
  62308988   62521981 (flat, cum) 74.97% of Total
         .          .    366:   }
         .          .    367: }
         .          .    368: if !isSlice &amp;&amp; t.IsVariadic() {
         .          .    369:   // prepare slice for remaining values
         .          .    370:   m := len(in) - n
         .      81921    371:   slice := MakeSlice(t.In(n), m, m)
         .          .    372:   elem := t.In(n).Elem()
         .          .    373:   for i := 0; i &lt; m; i++ {
         .          .    374:     x := in[n+i]
         .          .    375:     if xt := x.Type(); !xt.AssignableTo(elem) {
         .          .    376:       panic(&quot;reflect: cannot use &quot; + xt.String() + &quot; as type &quot; + elem.String() + &quot; in &quot; + op)
         .          .    377:     }
         .     131072    378:     slice.Index(i).Set(x)
         .          .    379:   }
         .          .    380:   origIn := in
         .          .    381:   in = make([]Value, n+1)
         .          .    382:   copy(in[:n], origIn)
         .          .    383:   in[n] = slice
         .          .    384: }
         .          .    385:
         .          .    386: nin := len(in)
         .          .    387: if nin != t.NumIn() {
         .          .    388:   panic(&quot;reflect.Value.Call: wrong argument count&quot;)
         .          .    389: }
         .          .    390: nout := t.NumOut()
         .          .    391:
         .          .    392: // Compute frame type, allocate a chunk of memory for frame
         .          .    393: frametype, _, retOffset, _ := funcLayout(t, rcvrtype)
     32769      32769    394: args := unsafe_New(frametype)
         .          .    395: off := uintptr(0)
         .          .    396:
         .          .    397: // Copy inputs into args.
         .          .    398: if rcvrtype != nil {
         .          .    399:   storeRcvr(rcvr, args)
         .          .    400:   off = ptrSize
         .          .    401: }
         .          .    402: for i, v := range in {
         .          .    403:   v.mustBeExported()
         .          .    404:   targ := t.In(i).(*rtype)
         .          .    405:   a := uintptr(targ.align)
         .          .    406:   off = (off + a - 1) &amp;^ (a - 1)
         .          .    407:   n := targ.size
         .          .    408:   addr := unsafe.Pointer(uintptr(args) + off)
         .          .    409:   v = v.assignTo(&quot;reflect.Value.Call&quot;, targ, addr)
         .          .    410:   if v.flag&amp;flagIndir != 0 {
         .          .    411:     memmove(addr, v.ptr, n)
         .          .    412:   } else {
         .          .    413:     *(*unsafe.Pointer)(addr) = v.ptr
         .          .    414:   }
         .          .    415:   off += n
         .          .    416: }
         .          .    417:
         .          .    418: // Call.
  62243451   62243451    419: call(fn, args, uint32(frametype.size), uint32(retOffset))
         .          .    420:
         .          .    421: // For testing; see TestCallMethodJump.
         .          .    422: if callGC {
         .          .    423:   runtime.GC()
         .          .    424: }
         .          .    425:
         .          .    426: // Copy return values out of args.
     32768      32768    427: ret := make([]Value, nout)
         .          .    428: off = retOffset
         .          .    429: for i := 0; i &lt; nout; i++ {
         .          .    430:   tv := t.Out(i)
         .          .    431:   a := uintptr(tv.Align())
         .          .    432:   off = (off + a - 1) &amp;^ (a - 1)

But all this stuff don't give a clue where are memory leaks in my code.

What does reflect.Value.call actually mean?
I don't remember that I used reflect.Value.call in my code.

答案1

得分: 4

gopkg.in/mgo.v2/bson使用反射来进行BSON的(解)编组。但是由于问题11786,实际导致内存泄漏的方法在性能分析中没有显示出来。

解决方法是向go tool pprof传递标志-runtime

英文:

Package gopkg.in/mgo.v2/bson uses reflection to (un)marshal BSON. But the actual method leaking memory is not shown in the profile because of issue 11786.

The workaround is to pass the flag -runtime to go tool pprof.

huangapple
  • 本文由 发表于 2016年2月18日 20:08:34
  • 转载请务必保留本文链接:https://go.coder-hub.com/35481090.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定