如何在golang中使用sync.Pool重用切片?

huangapple go评论85阅读模式
英文:

How to reuse slice with sync.Pool in golang?

问题

考虑以下代码:

type TestStruct struct {
	Name string
}

func TestSliceWithPool(t *testing.T) {
	var slicePool = sync.Pool{
		New: func() interface{} {
			t.Log("i am created")
			s := make([]interface{}, 0)
			return s
		},
	}

	s, _ := slicePool.Get().([]interface{})
	t.Logf("Lenth: %d, Cap: %d, Pointer: %p", len(s), cap(s), s)
	for i := 0; i < 9000; i++ {
		st := &TestStruct{Name: "test"}
		s = append(s, st)
	}
	for _, v := range s {
		if value, ok := v.(TestStruct); ok {
			if value.Name != "test" {
				t.Error("u are changed!")
			}
		}
	}
	s = s[:0]
	slicePool.Put(s)

	s2, _ := slicePool.Get().([]interface{})
	t.Logf("Lenth: %d, Cap: %d, Pointer: %p", len(s), cap(s), s)
	for i := 0; i < 8000; i++ {
		st := &TestStruct{Name: "test2"}
		s2 = append(s2, st)
	}
	for _, v := range s2 {
		if value, ok := v.(TestStruct); ok {
			if value.Name != "test2" {
				t.Error("u are changed!")
			}
		}
	}
	slicePool.Put(s2)
}

测试结果为:

slice_test.go:63: i am created
slice_test.go:70: Lenth: 0, Cap: 0, Pointer: 0x1019598
slice_test.go:86: Lenth: 0, Cap: 9728, Pointer: 0xc000500000

为什么只生成了一次,但地址不同?为什么容量是9728?我这样使用同一个切片有问题吗?

这段代码使用了sync.Pool来管理切片的重用。sync.Pool是一个用于存储临时对象的对象池,它可以减少对象的创建和垃圾回收的压力,提高性能。

在代码中,slicePool是一个sync.Pool对象,其中New字段指定了一个函数,用于创建新的切片。当从对象池中获取切片时,如果对象池为空,则会调用New函数创建一个新的切片。

在第一次调用slicePool.Get()时,由于对象池为空,所以会调用New函数创建一个新的切片,并将其返回。这个切片的地址是0x1019598。

在第二次调用slicePool.Get()时,由于对象池中已经有了一个切片,所以直接从对象池中取出并返回。这个切片的地址是0xc000500000。

至于为什么容量是9728,这是由于切片的扩容机制所决定的。切片在进行append操作时,如果容量不足,会自动进行扩容。扩容的策略是按照一定的规则进行的,具体规则可以参考Go语言的文档。在这段代码中,切片进行了多次append操作,导致容量逐渐增加,最终扩容到了9728。

至于使用同一个切片是否有问题,这取决于具体的业务逻辑。在这段代码中,切片在使用完毕后被放回了对象池,以便下次重用。这样可以减少内存分配和垃圾回收的开销,提高性能。但需要注意的是,在放回对象池之前,需要确保切片中的数据已经不再使用,否则可能会导致数据错乱或内存泄漏的问题。

英文:

Consider this code:

type TestStruct struct {
	Name string
}

func TestSliceWithPool(t *testing.T) {
	var slicePool = sync.Pool{
		New: func() interface{} {
			t.Log(&quot;i am created&quot;)
			s := make([]interface{}, 0)
			return s
		},
	}

	s, _ := slicePool.Get().([]interface{})
	t.Logf(&quot;Lenth: %d, Cap: %d, Pointer: %p&quot;, len(s), cap(s), s)
	for i := 0; i &lt; 9000; i++ {
		st := &amp;TestStruct{Name: &quot;test&quot;}
		s = append(s, st)
	}
	for _, v := range s {
		if value, ok := v.(TestStruct); ok {
			if value.Name != &quot;test&quot; {
				t.Error(&quot;u are changed!&quot;)
			}
		}
	}
	s = s[:0]
	slicePool.Put(s)

	s2, _ := slicePool.Get().([]interface{})
	t.Logf(&quot;Lenth: %d, Cap: %d, Pointer: %p&quot;, len(s), cap(s), s)
	for i := 0; i &lt; 8000; i++ {
		st := &amp;TestStruct{Name: &quot;test2&quot;}
		s2 = append(s2, st)
	}
	for _, v := range s2 {
		if value, ok := v.(TestStruct); ok {
			if value.Name != &quot;test2&quot; {
				t.Error(&quot;u are changed!&quot;)
			}
		}
	}
	slicePool.Put(s2)
}

The result of test is:

slice_test.go:63: i am created
slice_test.go:70: Lenth: 0, Cap: 0, Pointer: 0x1019598
slice_test.go:86: Lenth: 0, Cap: 9728, Pointer: 0xc000500000

Why is it generated only once but the address is different? And Why the cap is 9728?
Is there any problem when I use the same slice like this?

答案1

得分: 3

为什么它只生成一次,但地址不同?

因为在第一个for循环中,您将其追加到超出其容量的位置,而在New中,容量被设置为零,并且将其重新分配给append的结果。详情请参考:https://stackoverflow.com/questions/17395261/why-does-append-modify-the-provided-slice-see-example

如果我像这样使用相同的切片会有问题吗?

有可能会有问题。当您使用s = s[:0]重新切片s时,您重置了长度,但没有重置容量。底层数组仍然是之前追加和重新分配操作的那个数组。

因此,如果您再次追加到s2,容量将足够,不会导致重新分配,并且您将覆盖底层数组的前几个元素:

一个示例:

func TestSliceWithPool(t *testing.T) {
	var slicePool = sync.Pool{
		New: func() interface{} {
			t.Log("Created")
			s := make([]interface{}, 0)
			return s
		},
	}

	s, _ := slicePool.Get().([]interface{})
	for i := 0; i < 10; i++ {
		s = append(s, i)
	}
	fmt.Println(s) 
    // ^ 输出:[0 1 2 3 4 5 6 7 8 9]

	s = s[:0]
	slicePool.Put(s)

	s2, _ := slicePool.Get().([]interface{})
	fmt.Println(s)
    // ^ 输出:[]

	for i := 0; i < 5; i++ {
		s2 = append(s2, i*10)
	}
	fmt.Println(s2) 
    // ^ 输出:[0 10 20 30 40]

	fmt.Println(s2[:10])
    // ^ 输出:[0 10 20 30 40 5 6 7 8 9]
}

这可能是可以接受的,因为现在您有一个具有扩展容量的切片,不需要在追加时重新分配,但如果您的应用程序保留指向相同底层数组的其他切片头(如ss2)的情况下,这也可能是内存泄漏,从而阻止缓冲区的垃圾回收。

英文:

> Why is it generated only once but the address is different?

Because in the first for loop you append to it beyond its capacity, which in New is set to zero, and reassign to it the result of append. For details: https://stackoverflow.com/questions/17395261/why-does-append-modify-the-provided-slice-see-example

> Is there any problem when I use the same slice like this?

There could be. When you reslice s with s = s[:0], you are resetting the length but not the capacity. The backing array is still the same one from the previous append-and-reassign operation.

So if you append again to s2, the capacity will be enough to not cause reallocation, and you'll end up overwriting the first elements of the backing array:

A demonstrative example:

func TestSliceWithPool(t *testing.T) {
	var slicePool = sync.Pool{
		New: func() interface{} {
			t.Log(&quot;Created&quot;)
			s := make([]interface{}, 0)
			return s
		},
	}

	s, _ := slicePool.Get().([]interface{})
	for i := 0; i &lt; 10; i++ {
		s = append(s, i)
	}
	fmt.Println(s) 
    // ^ output: [0 1 2 3 4 5 6 7 8 9]

	s = s[:0]
	slicePool.Put(s)

	s2, _ := slicePool.Get().([]interface{})
	fmt.Println(s)
    // ^ output: []

	for i := 0; i &lt; 5; i++ {
		s2 = append(s2, i*10)
	}
	fmt.Println(s2) 
    // ^ output: [0 10 20 30 40]

	fmt.Println(s2[:10])
    // ^ output: [0 10 20 30 40 5 6 7 8 9]
}

This might be okay, since you now have a slice with extended capacity that doesn't need reallocation on append, but it could also be a memory leak if your application keeps around other slice headers pointing to the same backing array (as in case of s and s2), thus preventing garbage collection of the buffers.

huangapple
  • 本文由 发表于 2022年7月7日 14:19:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/72893095.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定