在Go语言程序中查找内存泄漏 – reflect.Value.call的含义

huangapple
go评论122阅读模式
英文:

finding memory leaks in golang program - reflect.Value.call meaning

问题

我正在尝试找到一个内存泄漏的代码片段。

在启动全新的Web应用程序后，它占用6 MB的内存。
经过大约12,000个请求后，它占用28 MB的内存。

我在启动后保存了它的堆内存快照：

  1. curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/6mb.heap

然后在12,000个请求后保存了堆内存快照：

  1. curl -s localhost:6060/debug/pprof/heap > ~/debug/heavyHeap/28mb.heap

然后我尝试查看分配的对象差异：

  1. go tool pprof -alloc_objects -base ~/debug/heavyHeap/6mb.heap $GOPATH/myBin ~/debug/heavyHeap/28mb.heap

运行top命令：

  1. 进入交互模式（输入"help"获取命令列表）
  2. (pprof) top
  3. 总共 83397023 个对象中的 73949086 个（占总数的 88.67%）
  4. 删除了 299 个节点（累计 <= 416985）
  5. 显示前 10 个节点（累计 >= 1802254）
  6.       flat  flat%   sum%        cum   cum%
  7.   62308988 74.71% 74.71%   62521981 74.97%  reflect.Value.call
  8.    2413961  2.89% 77.61%    2424884  2.91%  calldb.fromToDiff
  9.    1769493  2.12% 79.73%    3796564  4.55%  gopkg.in/mgo.v2/bson.(*decoder).readElemTo
  10.    1622034  1.94% 81.67%    1622034  1.94%  gopkg.in/mgo.v2/bson.(*decoder).readCStr
  11.    1270739  1.52% 83.20%    1401813  1.68%  reflect.(*structType).FieldByNameFunc
  12.    1130028  1.35% 84.55%    1130028  1.35%  reflect.Value.MapKeys
  13.     933704  1.12% 85.67%     933704  1.12%  gopkg.in/mgo.v2/bson.(*decoder).readStr
  14.     927261  1.11% 86.79%     946166  1.13%  fmt.Sprintf
  15.     819209  0.98% 87.77%    1119590  1.34%  my.AnchorWithClassAndDisabledAndStyle

我列出了最重的项 reflect.Value.call：

  1. (pprof) list reflect.Value.call
  2. 总共: 83397023
  3. ROUTINE ======================== reflect.Value.call in /usr/local/go/src/reflect/value.go
  4.   62308988   62521981 (flat, cum) 占总数的 74.97%
  5.          .          .    366:   }
  6.          .          .    367: }
  7.          .          .    368: if !isSlice && t.IsVariadic() {
  8.          .          .    369:   // prepare slice for remaining values
  9.          .          .    370:   m := len(in) - n
  10.          .      81921    371:   slice := MakeSlice(t.In(n), m, m)
  11.          .          .    372:   elem := t.In(n).Elem()
  12.          .          .    373:   for i := 0; i < m; i++ {
  13.          .          .    374:     x := in[n+i]
  14.          .          .    375:     if xt := x.Type(); !xt.AssignableTo(elem) {
  15.          .          .    376:       panic("reflect: cannot use " + xt.String() + " as type " + elem.String() + " in " + op)
  16.          .          .    377:     }
  17.          .     131072    378:     slice.Index(i).Set(x)
  18.          .          .    379:   }
  19.          .          .    380:   origIn := in
  20.          .          .    381:   in = make([]Value, n+1)
  21.          .          .    382:   copy(in[:n], origIn)
  22.          .          .    383:   in[n] = slice
  23.          .          .    384: }
  24.          .          .    385:
  25.          .          .    386: nin := len(in)
  26.          .          .    387: if nin != t.NumIn() {
  27.          .          .    388:   panic("reflect.Value.Call: wrong argument count")
  28.          .          .    389: }
  29.          .          .    390: nout := t.NumOut()
  30.          .          .    391:
  31.          .          .    392: // Compute frame type, allocate a chunk of memory for frame
  32.          .          .    393: frametype, _, retOffset, _ := funcLayout(t, rcvrtype)
  33.      32769      32769    394: args := unsafe_New(frametype)
  34.          .          .    395: off := uintptr(0)
  35.          .          .    396:
  36.          .          .    397: // Copy inputs into args.
  37.          .          .    398: if rcvrtype != nil {
  38.          .          .    399:   storeRcvr(rcvr, args)
  39.          .          .    400:   off = ptrSize
  40.          .          .    401: }
  41.          .          .    402: for i, v := range in {
  42.          .          .    403:   v.mustBeExported()
  43.          .          .    404:   targ := t.In(i).(*rtype)
  44.          .          .    405:   a := uintptr(targ.align)
  45.          .          .    406:   off = (off + a - 1) &^ (a - 1)
  46.          .          .    407:   n := targ.size
  47.          .          .    408:   addr := unsafe.Pointer(uintptr(args) + off)
  48.          .          .    409:   v = v.assignTo("reflect.Value.Call", targ, addr)
  49.          .          .    410:   if v.flag&flagIndir != 0 {
  50.          .          .    411:     memmove(addr, v.ptr, n)
  51.          .          .    412:   } else {
  52.          .          .    413:     *(*unsafe.Pointer)(addr) = v.ptr
  53.          .          .    414:   }
  54.          .          .    415:   off += n
  55.          .          .    416: }
  56.          .          .    417:
  57.          .          .    418: // Call.
  58.   62243451   62243451    419: call(fn, args, uint32(frametype.size), uint32(retOffset))
  59.          .          .    420:
  60.          .          .    421: // For testing; see TestCallMethodJump.
  61.          .          .    422: if callGC {
  62.          .          .    423:   runtime.GC()
  63.          .          .    424: }
  64.          .          .    425:
  65.          .          .    426: // Copy return values out of args.
  66.      32768      32768    427: ret := make([]Value, nout)
  67.          .          .    428: off = retOffset
  68.          .          .    429: for i := 0; i < nout; i++ {
  69.          .          .    430:   tv := t.Out(i)
  70.          .          .    431:   a := uintptr(tv.Align())
  71.          .          .    432:   off = (off + a - 1) &^ (a - 1)

但是所有这些信息都没有给出我代码中的内存泄漏位置的线索。

reflect.Value.call实际上是什么意思？
我不记得在我的代码中使用过reflect.Value.call。

英文:

I'm trying to find piece of code which is memory leaking.

After launching fresh web application, it's 6 MB.
After about 12k requests, it's 28 MB.

I saved its heap just after launching

  1. curl -s localhost:6060/debug/pprof/heap &gt; ~/debug/heavyHeap/6mb.heap

And after 12k requests:

  1. curl -s localhost:6060/debug/pprof/heap &gt; ~/debug/heavyHeap/28mb.heap

Then I am trying to see allocated objects difference:

  1. go tool pprof -alloc_objects -base ~/debug/heavyHeap/6mb.heap $GOPATH/myBin ~/debug/heavyHeap/28mb.heap

Run top command:

  1. Entering interactive mode (type &quot;help&quot; for commands)
  2. (pprof) top
  3. 73949086 of 83397023 total (88.67%)
  4. Dropped 299 nodes (cum &lt;= 416985)
  5. Showing top 10 nodes out of 117 (cum &gt;= 1802254)
  6.       flat  flat%   sum%        cum   cum%
  7.   62308988 74.71% 74.71%   62521981 74.97%  reflect.Value.call
  8.    2413961  2.89% 77.61%    2424884  2.91%  calldb.fromToDiff
  9.    1769493  2.12% 79.73%    3796564  4.55%  gopkg.in/mgo.v2/bson.(*decoder).readElemTo
  10.    1622034  1.94% 81.67%    1622034  1.94%  gopkg.in/mgo.v2/bson.(*decoder).readCStr
  11.    1270739  1.52% 83.20%    1401813  1.68%  reflect.(*structType).FieldByNameFunc
  12.    1130028  1.35% 84.55%    1130028  1.35%  reflect.Value.MapKeys
  13.     933704  1.12% 85.67%     933704  1.12%  gopkg.in/mgo.v2/bson.(*decoder).readStr
  14.     927261  1.11% 86.79%     946166  1.13%  fmt.Sprintf
  15.     819209  0.98% 87.77%    1119590  1.34%  my.AnchorWithClassAndDisabledAndStyle

I list the heaviest item reflect.Value.call:

  1. (pprof) list reflect.Value.call
  2. Total: 83397023
  3. ROUTINE ======================== reflect.Value.call in /usr/local/go/src/reflect/value.go
  4.   62308988   62521981 (flat, cum) 74.97% of Total
  5.          .          .    366:   }
  6.          .          .    367: }
  7.          .          .    368: if !isSlice &amp;&amp; t.IsVariadic() {
  8.          .          .    369:   // prepare slice for remaining values
  9.          .          .    370:   m := len(in) - n
  10.          .      81921    371:   slice := MakeSlice(t.In(n), m, m)
  11.          .          .    372:   elem := t.In(n).Elem()
  12.          .          .    373:   for i := 0; i &lt; m; i++ {
  13.          .          .    374:     x := in[n+i]
  14.          .          .    375:     if xt := x.Type(); !xt.AssignableTo(elem) {
  15.          .          .    376:       panic(&quot;reflect: cannot use &quot; + xt.String() + &quot; as type &quot; + elem.String() + &quot; in &quot; + op)
  16.          .          .    377:     }
  17.          .     131072    378:     slice.Index(i).Set(x)
  18.          .          .    379:   }
  19.          .          .    380:   origIn := in
  20.          .          .    381:   in = make([]Value, n+1)
  21.          .          .    382:   copy(in[:n], origIn)
  22.          .          .    383:   in[n] = slice
  23.          .          .    384: }
  24.          .          .    385:
  25.          .          .    386: nin := len(in)
  26.          .          .    387: if nin != t.NumIn() {
  27.          .          .    388:   panic(&quot;reflect.Value.Call: wrong argument count&quot;)
  28.          .          .    389: }
  29.          .          .    390: nout := t.NumOut()
  30.          .          .    391:
  31.          .          .    392: // Compute frame type, allocate a chunk of memory for frame
  32.          .          .    393: frametype, _, retOffset, _ := funcLayout(t, rcvrtype)
  33.      32769      32769    394: args := unsafe_New(frametype)
  34.          .          .    395: off := uintptr(0)
  35.          .          .    396:
  36.          .          .    397: // Copy inputs into args.
  37.          .          .    398: if rcvrtype != nil {
  38.          .          .    399:   storeRcvr(rcvr, args)
  39.          .          .    400:   off = ptrSize
  40.          .          .    401: }
  41.          .          .    402: for i, v := range in {
  42.          .          .    403:   v.mustBeExported()
  43.          .          .    404:   targ := t.In(i).(*rtype)
  44.          .          .    405:   a := uintptr(targ.align)
  45.          .          .    406:   off = (off + a - 1) &amp;^ (a - 1)
  46.          .          .    407:   n := targ.size
  47.          .          .    408:   addr := unsafe.Pointer(uintptr(args) + off)
  48.          .          .    409:   v = v.assignTo(&quot;reflect.Value.Call&quot;, targ, addr)
  49.          .          .    410:   if v.flag&amp;flagIndir != 0 {
  50.          .          .    411:     memmove(addr, v.ptr, n)
  51.          .          .    412:   } else {
  52.          .          .    413:     *(*unsafe.Pointer)(addr) = v.ptr
  53.          .          .    414:   }
  54.          .          .    415:   off += n
  55.          .          .    416: }
  56.          .          .    417:
  57.          .          .    418: // Call.
  58.   62243451   62243451    419: call(fn, args, uint32(frametype.size), uint32(retOffset))
  59.          .          .    420:
  60.          .          .    421: // For testing; see TestCallMethodJump.
  61.          .          .    422: if callGC {
  62.          .          .    423:   runtime.GC()
  63.          .          .    424: }
  64.          .          .    425:
  65.          .          .    426: // Copy return values out of args.
  66.      32768      32768    427: ret := make([]Value, nout)
  67.          .          .    428: off = retOffset
  68.          .          .    429: for i := 0; i &lt; nout; i++ {
  69.          .          .    430:   tv := t.Out(i)
  70.          .          .    431:   a := uintptr(tv.Align())
  71.          .          .    432:   off = (off + a - 1) &amp;^ (a - 1)

But all this stuff don't give a clue where are memory leaks in my code.

What does reflect.Value.call actually mean?
I don't remember that I used reflect.Value.call in my code.

答案1

得分: 4

包gopkg.in/mgo.v2/bson使用反射来进行BSON的(解)编组。但是由于问题11786，实际导致内存泄漏的方法在性能分析中没有显示出来。

解决方法是向go tool pprof传递标志-runtime。

英文:

Package gopkg.in/mgo.v2/bson uses reflection to (un)marshal BSON. But the actual method leaking memory is not shown in the profile because of issue 11786.

The workaround is to pass the flag -runtime to go tool pprof.

huangapple
  • 本文由 发表于 2016年2月18日 20:08:34
  • 转载请务必保留本文链接：https://go.coder-hub.com/35481090.html
  • go
  • memory-leaks
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定