将`interface{}`转换回切片为什么会导致额外的堆分配?

huangapple go评论68阅读模式
英文:

Why does converting `interface{}` back to slice cause an additional heap allocation?

问题

func BenchmarkPool(b *testing.B) {
	b.ReportAllocs()
	p := sync.Pool{New: func() interface{} {
		return make([]byte, 1024)
	}}
	for i := 0; i < b.N; i++ {
		bts := p.Get().([]byte)
		p.Put(bts)
	}
}

这个基准测试在 go1.19.5 中的输出如下:

BenchmarkPool
BenchmarkPool-10    	47578498	        24.47 ns/op	      24 B/op	       1 allocs/op

当使用 *[]byte 时,情况就不同了:

func BenchmarkPool(b *testing.B) {
	b.ReportAllocs()
	p := sync.Pool{New: func() interface{} {
		bts := make([]byte, 1024)
		return &bts
	}}
	for i := 0; i < b.N; i++ {
		bts := p.Get().(*[]byte)
		p.Put(bts)
	}
}
BenchmarkPool
BenchmarkPool-10    	142008002	         8.581 ns/op	       0 B/op	       0 allocs/op

似乎将 interface{} 转换回切片会导致额外的堆分配。
为什么 Go 需要这个额外的分配?在其中的设计考虑是什么?

英文:
func BenchmarkPool(b *testing.B) {
	b.ReportAllocs()
	p := sync.Pool{New: func() interface{} {
		return make([]byte, 1024)
	}}
	for i := 0; i &lt; b.N; i++ {
		bts := p.Get().([]byte)
		p.Put(bts)
	}
}

This benchmark gives the following output in go1.19.5.

BenchmarkPool
BenchmarkPool-10    	47578498	        24.47 ns/op	      24 B/op	       1 allocs/op

When use *[]byte, things get different:

func BenchmarkPool(b *testing.B) {
	b.ReportAllocs()
	p := sync.Pool{New: func() interface{} {
		bts := make([]byte, 1024)
		return &amp;bts
	}}
	for i := 0; i &lt; b.N; i++ {
		bts := p.Get().(*[]byte)
		p.Put(bts)
	}
}
BenchmarkPool
BenchmarkPool-10    	142008002	         8.581 ns/op	       0 B/op	       0 allocs/op

It seems that convert interface{} back to slice cause an additional heap allocation.
Why does Go need this additional allocation? What's the design consideration under it?

答案1

得分: 1

一个切片在"动态"意义上是指根据需要分配和丢弃头部和后备数组。当你将[]byte值放入池中时,切片头部总是被复制的,所以实际上只有后备数组被重用,而每次将切片返回到池中时,你需要为头部分配额外的24字节。换句话说,当你只传入一个副本时,你不能重用切片头部。

如果你希望池存储和重用整个切片值,那么你需要使用*[]byte,因为你需要一个指向要存储在池中的值的指针。这还允许你将不同大小的切片值返回到池中(可能使用append进行扩展),这可能对你的用例是否合适取决于。

如果你只想要一个静态大小的缓冲区进行重用,那么你需要的是一个数组,或者更具体地说,一个指向数组的指针,这样更清楚地说明了对池化值的期望。

func BenchmarkPool(b *testing.B) {
	b.ReportAllocs()
	p := sync.Pool{New: func() interface{} {
		return &[1024]byte{}
	}}
	for i := 0; i < b.N; i++ {
		buff := p.Get().(*[1024]byte)
		p.Put(buff)
	}
}

你可以根据需要分配一个围绕该数组的切片,并在将其返回到池中时将其转换回数组。

英文:

A slice is meant to be "dynamic" in the sense that you allocate and discard the header and backing array as needed. When you pool the []byte value, the slice header is always copied, so all that is essentially reused is the backing array, and you need to allocate the extra 24 bytes for the header every time to return the slice to the pool. In other words, you can't reuse the slice header when you are only passing in a copy.

If you want the pool to store and reuse an entire slice value, then you want to use *[]byte because you need a pointer to the value you want to store in the pool. This would also allow you to return a differently sized slice value to the pool (possibly enlarged with append), which may or may not be desirable for your use case.

If you want only a statically sized buffer for reuse, then what you want is an array, or more specifically, a pointer to an array, which makes it much clearer what the expectations are of the pooled value.

func BenchmarkPool(b *testing.B) {
	b.ReportAllocs()
	p := sync.Pool{New: func() interface{} {
		return &amp;[1024]byte{}
	}}
	for i := 0; i &lt; b.N; i++ {
		buff := p.Get().(*[1024]byte)
		p.Put(buff)
	}
}

You can allocate a slice around that array as needed, and convert it back to an array when returning it to the pool.

答案2

得分: 0

不是将any转换为[]byte导致了分配,而是将[]byte转换为any导致了分配。在将bts传递给(*sync.Pool).Put之前,p.Put(bts)会隐式地将参数bts转换为any。在GoGC 1.19中,接口被实现为一对指针,一个指向类型元数据,一个指向实际对象。在这种情况下,第二个指针逃逸到池中,导致分配了切片对象。这不仅适用于切片类型,还适用于任何其他非指针类型。

对于指针类型,比如*[]byte,编译器会执行一种优化,直接将其值放入iface结构中,这样在转换为接口时就不会分配*[]byte实例。因此,通常建议将指针放入池中,而不是结构本身。

英文:

It is not the conversion of any to []byte that causes allocation, it is the conversion of []byte to any that does. p.Put(bts) implicitly casts the argument bts to any before passing it to (*sync.Pool).Put. An interface as in GoGC 1.19 is implemented as a pair of pointers, one to the type metadata and one to the actual object, and it is the second pointer that escapes to the pool in this case that results in the slice object being allocated. This does not only hold for slice types but also any other non-pointer types.

For a pointer, such as a *[]byte, the compiler performs an optimisation that put its value directly into the iface struct, which removes the allocation of a *[]byte instance when converting to an interface. As a result, it is often recommended to put the pointer in pools instead of the struct themselves.

huangapple
  • 本文由 发表于 2023年3月14日 17:05:34
  • 转载请务必保留本文链接:https://go.coder-hub.com/75731004.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定