英文:
Go Interface memory leakage
问题
似乎Go语言没有正确释放基于接口的指针。
在这段代码中,第一次使用i:=&Implementation{}; i.Method(data)
的语法时,Go语言会释放指针,但是第二次使用var i Interface; i=&Implementation{}; i.Method(data)
的语法时,Go语言可能没有正确处理内存。我在代码中展示了好的和坏的语法,两种语法有什么区别?为什么第二行会浪费内存?
- 第一种语法:
i:=&Implementation{}; i.Method(data)
- 第二种语法:
var i Interface; i=&Implementation{}; i.Method(data)
此外,输出结果显示了问题:
First Memory Usage: 0MB, expected 0
Second Memory Usage: 27MB, while expected 0 //虽然数字可能不同,但差异显著
我期望第一次和第二次尝试之间的结果是相同的。
英文:
seemly Go doesn't release interface based pointers properly.
package main //interface memory leakage
import (
"fmt"
"runtime"
)
type Interface interface{ Method(v any) }
type Implementation struct{}
func (i *Implementation) Method(v any) {}
func main() {
pages := Pages()
fmt.Printf("First Memory Usage: %dMB, expected 0\n", good()) // i:=&Implementation{}; i.Method(data)
fmt.Printf("Second Memory Usage: %dMB, while expected 0\n", bad()) // var i Interface; i=&Implementation{}; i.Method(data)
var xor byte //do somthing
for j := 0; j < NUM_OF_PAGES; j++ {
xor ^= pages[j][PAGE_SIZE-1]
}
print("xor: ", xor)
}
var data = "123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_"
var LOOP_SIZE = 3 * 1000 * 1000
const NUM_OF_PAGES = 10
const PAGE_SIZE = 1024 * 1024
func Pages() [][]byte {
var pages [][]byte
for j := 0; j < NUM_OF_PAGES; j++ {
p := make([]byte, PAGE_SIZE)
for i := 0; i < len(p); i++ {
p[i] = 'A'
}
pages = append(pages, p)
}
return pages
}
func good() int64 {
var m0, m1 runtime.MemStats
runtime.ReadMemStats(&m0)
i := &Implementation{} //good line
for n := 0; n < LOOP_SIZE; n++ {
i.Method(data)
}
//runtime.GC()
runtime.ReadMemStats(&m1)
return int64(m1.Alloc-m0.Alloc) / 1024 / 1024
}
func bad() int64 {
var m0, m1 runtime.MemStats
runtime.ReadMemStats(&m0)
var i Interface //bad line
i = &Implementation{} //bad line
for n := 0; n < LOOP_SIZE; n++ {
i.Method(data)
}
//runtime.GC()
runtime.ReadMemStats(&m1)
return int64(m1.Alloc-m0.Alloc) / 1024 / 1024
}
https://go.dev/play/p/0zlbPxs6WCV
here the code is same, first time Go release pointers but second time, when we are using interface based pointer likely go doesn't handle memory properly.
I showed the good and bad syntax within the code, what's difference between two syntax? why second line wastes memory()?
- first:
i:=&Implementation{}; i.Method(data)
- second:
var i Interface; i=&Implementation{}; i.Method(data)
Also the output shows the problem:
First Memory Usage: 0MB, expected 0
Second Memory Usage: 27MB, while expected 0 //However the number is variable but almost is different significantly
I expect same result between first and second try.
答案1
得分: 2
i := &Implementation{}
第一个实现是指向直接类型的指针。
不需要进行额外的工作,因为在编译时已经知道需要在实例上调用哪个方法。
var i Interface
i = &Implementation{}
第二个实现将指针封装为接口。
因为i
的值可以是满足接口的任何值,所以不知道需要执行哪个方法,因此需要在该变量中存储更多的信息。
当将指针封装为接口时,必须同时存储指针的类型和值。因此,必须为该额外的间接层进行额外的分配。分配是在堆上进行还是优化到栈上是由逃逸分析决定的。
英文:
i := &Implementation{}
The first implementation is a pointer to a direct type.
No additional work needs to be done, because at compile time it is known which method needs to be invoked on the instance.
var i Interface
i = &Implementation{}
The second implementation boxes the pointer as an interface.
Because the value of i
could be any value that satisfies the interface, it is not known which method needs to be executed, so more information needs to be stored in that variable.
When a pointer is boxed as an interface, it must store both the type of the pointer and the value of the pointer. So, an additional allocation must occur for that additional layer of indirection. Whether allocations are made on the heap or optimized onto the stack is determined by escape analysis.
答案2
得分: 2
让我们首先澄清一下,你的程序中没有内存泄漏——与你的“好”案例相比,你的“坏”案例中有额外的堆分配,但在进行垃圾回收后,所有这些内存都会被释放。这意味着没有泄漏。
但是肯定有更多的堆分配。
有两种方法可以使你的“坏”案例与“好”案例一样多地使用堆:
-
将
data
变量声明为any
类型:var data any = "123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_"
-
将方法的参数类型声明为
string
:type Interface interface{ Method(v string) } type Implementation struct{} func (i *Implementation) Method(v string) {}
在上述两种情况下,你消除了每次调用方法时将字符串转换为接口的需要,因此编译器不会为每次调用执行堆分配。
“坏”案例分配更多堆的真正原因是,在“好”案例中,Go编译器知道确切的方法实现并进行了分析。它看到该方法什么也不做,并完全避免了对该方法的调用。在“坏”案例中,它看到对接口的方法调用,静态分析不考虑在该点唯一可能的实现是结构体Implementation
,因此无法确定该方法调用没有任何效果。
如果你更改Method
的实现以对参数执行某些操作,例如将其存储在包范围的变量中,那么编译器无法在“好”中优化方法调用,并且“好”和“坏”都将在堆上分配相同的数量。
英文:
Let's start by clarifying that there is no memory leak in your program - there are extra heap allocations in your bad
case compared to your good
case, but after a garbage collection, all that memory is released. That means there is no leak.
But there are certainly more heap allocations.
There are two ways to make your bad
case have as much heap usage as the good
case:
-
declare the
data
variable as typeany
:var data any = "123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_123456789_"
-
declare the argument type of the method as
string
:type Interface interface{ Method(v string) } type Implementation struct{} func (i *Implementation) Method(v string) {}
In both above cases, you eliminate the need to convert a string to and interface for every invocation of the method, so the compiler will not perform a heap allocation per invocation.
The real reason that the bad
case allocates more on the heap, is that in the good
case, the Go compiler knows the exact method implementation, and analysis it. It sees that the method does nothing, and avoids the call to the method altogether. In the bad
case, it sees a method call on an interface, and the static analysis doesn't consider that the only possible implementation at that point is the struct Implementation
, so it cannot determine that the method call doesn't have any effect.
If you change the implementation of Method
to do something with the argument, like storing it in a package-scoped variable, then the compiler cannot optimise the method call away in good
, and good
and bad
will both allocate the same amount on the heap.
答案3
得分: 0
如前所述,没有内存泄漏 - 只有额外的堆分配。
Go编译器无法预测接口、切片或映射变量的实际大小,因此它在堆中而不是栈中为其分配内存。请注意,Go中的字符串是一个只读的字节切片。
Rus Cox在Go中有一篇关于接口的很棒的文章
https://research.swtch.com/interfaces
此外,您可能对William Kennedy的一系列文章感兴趣
https://www.ardanlabs.com/blog/2017/05/language-mechanics-on-stacks-and-pointers.html
英文:
As mentioned earlier, there is no memory leak - there are extra heap allocations.
Go compiler cannot predict the actual size of an interface, slice or map variable, so it allocates memory for it in the heap and not in the stack. Take a note, that a string in Go is a read-only slice of bytes.
Rus Cox has a great post about interfaces in Go
https://research.swtch.com/interfaces
Also you may be interested in William Kennedy's series of posts
https://www.ardanlabs.com/blog/2017/05/language-mechanics-on-stacks-and-pointers.html
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论