追加操作是否线程安全?

huangapple go评论78阅读模式
英文:

Append not thread-safe?

问题

我注意到,如果我尝试在for循环内使用goroutine来追加到一个切片中,会出现丢失/空白数据的情况:

destSlice := make([]myClass, 0)

var wg sync.WaitGroup
for _, myObject := range sourceSlice {
    wg.Add(1)
    go func(closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName
        destSlice = append(destSlice, tmpObj)
    }(myObject)
}
wg.Wait()

有时,当我打印destSlice中的所有AttributeName时,有些元素是空字符串(""),而其他时候,sourceSlice中的一些元素在destSlice中不存在。

我的代码是否存在数据竞争问题?这是否意味着append在多个goroutine并发使用时不是线程安全的?

英文:

I noticed that if I tried appending to a slice using goroutines inside a for loop, there would be instances where I would get missing/blank data:

destSlice := make([]myClass, 0)

var wg sync.WaitGroup
for _, myObject := range sourceSlice {
    wg.Add(1)
    go func(closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName
        destSlice = append(destSlice, tmpObj)
    }(myObject)
}
wg.Wait()

Sometimes, when I print all AttributeNames from destSlice, some elements are empty strings (""), and other times, some elements from sourceSlice are not present in destSlice.

Does my code have a data race, and does this mean that append is not thread-safe for concurrent use by multiple goroutines?

答案1

得分: 55

在Go语言中,没有任何值可以安全地进行并发读写,切片(即slice headers)也不例外。

是的,你的代码存在数据竞争。可以使用-race选项运行代码进行验证。

示例:

type myClass struct {
    AttributeName string
}
sourceSlice := make([]myClass, 100)

destSlice := make([]myClass, 0)

var wg sync.WaitGroup
for _, myObject := range sourceSlice {
    wg.Add(1)
    go func(closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName
        destSlice = append(destSlice, tmpObj)
    }(myObject)
}
wg.Wait()

使用以下命令运行代码:

go run -race play.go

输出结果为:

==================
WARNING: DATA RACE
Read at 0x00c420074000 by goroutine 6:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0x69

Previous write at 0x00c420074000 by goroutine 5:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0x106

Goroutine 6 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb

Goroutine 5 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb
==================
==================
WARNING: DATA RACE
Read at 0x00c42007e000 by goroutine 6:
  runtime.growslice()
      /usr/local/go/src/runtime/slice.go:82 +0x0
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0x1a7

Previous write at 0x00c42007e000 by goroutine 5:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0xc4

Goroutine 6 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb

Goroutine 5 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb
==================
==================
WARNING: DATA RACE
Write at 0x00c420098120 by goroutine 80:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0xc4

Previous write at 0x00c420098120 by goroutine 70:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0xc4

Goroutine 80 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb

Goroutine 70 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb
==================
Found 3 data race(s)
exit status 66

解决方法很简单,使用sync.Mutex来保护对destSlice值的写入:

var (
    mu        = &sync.Mutex{}
    destSlice = make([]myClass, 0)
)

var wg sync.WaitGroup
for _, myObject := range sourceSlice {
    wg.Add(1)
    go func(closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName
        mu.Lock()
        destSlice = append(destSlice, tmpObj)
        mu.Unlock()
    }(myObject)
}
wg.Wait()

你也可以用其他方法解决这个问题,例如可以使用一个通道,在通道上发送要追加的值,并有一个指定的goroutine从该通道接收并进行追加操作。

还要注意,虽然切片头部不安全,但切片元素作为不同的变量,可以在没有同步的情况下并发写入(因为它们是不同的变量)。参见这里

英文:

In Go no value is safe for concurrent read/write, slices (which are slice headers) are no exception.

Yes, your code has data races. Run with the -race option to verify.

Example:

type myClass struct {
	AttributeName string
}
sourceSlice := make([]myClass, 100)

destSlice := make([]myClass, 0)

var wg sync.WaitGroup
for _, myObject := range sourceSlice {
	wg.Add(1)
	go func(closureMyObject myClass) {
		defer wg.Done()
		var tmpObj myClass
		tmpObj.AttributeName = closureMyObject.AttributeName
		destSlice = append(destSlice, tmpObj)
	}(myObject)
}
wg.Wait()

Running it with

go run -race play.go

Output is:

==================
WARNING: DATA RACE
Read at 0x00c420074000 by goroutine 6:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0x69

Previous write at 0x00c420074000 by goroutine 5:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0x106

Goroutine 6 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb

Goroutine 5 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb
==================
==================
WARNING: DATA RACE
Read at 0x00c42007e000 by goroutine 6:
  runtime.growslice()
      /usr/local/go/src/runtime/slice.go:82 +0x0
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0x1a7

Previous write at 0x00c42007e000 by goroutine 5:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0xc4

Goroutine 6 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb

Goroutine 5 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb
==================
==================
WARNING: DATA RACE
Write at 0x00c420098120 by goroutine 80:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0xc4

Previous write at 0x00c420098120 by goroutine 70:
  main.main.func1()
      /home/icza/gows/src/play/play.go:20 +0xc4

Goroutine 80 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb

Goroutine 70 (running) created at:
  main.main()
      /home/icza/gows/src/play/play.go:21 +0x1cb
==================
Found 3 data race(s)
exit status 66

Solution is simple, use a sync.Mutex to protect writing the destSlice value:

var (
    mu        = &sync.Mutex{}
    destSlice = make([]myClass, 0)
)

var wg sync.WaitGroup
for _, myObject := range sourceSlice {
	wg.Add(1)
	go func(closureMyObject myClass) {
		defer wg.Done()
		var tmpObj myClass
		tmpObj.AttributeName = closureMyObject.AttributeName
		mu.Lock()
		destSlice = append(destSlice, tmpObj)
		mu.Unlock()
	}(myObject)
}
wg.Wait()

You could also solve it in other ways, e.g. you could use a channel on which you'd send the value to be appended, and have a designated goroutine receiving from this channel and do the append.

Also note that while slice headers are not safe, slice elements act as different variables and different slice elements can be written concurrently without synchronization (because they are distinct variables). See https://stackoverflow.com/questions/49879322/can-i-concurrently-write-different-slice-elements/49879469#49879469

答案2

得分: 17

这是一个相当古老的问题,但有另一个小的改进可以帮助摆脱互斥锁。你可以使用索引来添加到数组中,每个Go协程将使用自己的索引。在这种情况下,不需要同步。

destSlice := make([]myClass, len(sourceSlice))

var wg sync.WaitGroup
for i, myObject := range sourceSlice {
    wg.Add(1)
    go func(idx int, closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName

        destSlice[idx] = tmpObj
     }(i, myObject)
}
wg.Wait()

这段代码中,我们创建了一个目标切片 destSlice,长度与源切片 sourceSlice 相同。然后,我们使用 sync.WaitGroup 来等待所有的协程完成。

在每个循环迭代中,我们使用 go 关键字启动一个匿名函数作为一个新的协程。这个匿名函数接受两个参数:idx 表示索引,closureMyObject 表示当前迭代的对象。在匿名函数中,我们创建了一个临时对象 tmpObj,并将闭包对象的 AttributeName 赋值给它。最后,我们将临时对象赋值给目标切片的对应索引位置。

通过使用每个协程自己的索引,我们避免了对目标切片的并发访问,从而避免了同步操作的需要。最后,我们使用 wg.Wait() 等待所有的协程完成。

英文:

It's quite an old question but there is another small improvement that helps to get rid of mutex. You can use index to add to array. Each go routine will use it's own index. In this case synchronization is not necessary.

destSlice := make([]myClass, len(sourceSlice))

var wg sync.WaitGroup
for i, myObject := range sourceSlice {
    wg.Add(1)
    go func(idx int, closureMyObject myClass) {
        defer wg.Done()
        var tmpObj myClass
        tmpObj.AttributeName = closureMyObject.AttributeName

        destSlice[idx] = tmpObj
     }(i, myObject)
}
wg.Wait()

答案3

得分: 6

为了给这个问题提供一个更近期的解决方案,看起来Go语言发布了一个用于同步目的的新地图:

https://godoc.org/golang.org/x/sync/syncmap

英文:

To give a more recent solution to this problem, it looks like Go has released a new map for sync purposes:

https://godoc.org/golang.org/x/sync/syncmap

答案4

得分: 6

问题已经回答了,但我最喜欢解决这个问题的方式是使用errgroup。文档中的一个示例正是这个问题,还增加了错误处理的部分。

下面是文档中示例的核心代码:

g, ctx := errgroup.WithContext(ctx)

searches := []Search{Web, Image, Video}
results := make([]Result, len(searches))
for i, search := range searches {
    i, search := i, search // https://golang.org/doc/faq#closures_and_goroutines
    g.Go(func() error {
        result, err := search(ctx, query)
        if err == nil {
            results[i] = result
        }
        return err
    })
}
if err := g.Wait(); err != nil {
    return nil, err
}
return results, nil

希望对那些不熟悉errgroup包的人有所帮助。

英文:

Question has been answered, but my favorite way to solve this problem is with errgroup. One of the examples in the docs is this exact problem plus one nice addition the handling of errors.

Below is the meat of the example from the docs:

g, ctx := errgroup.WithContext(ctx)

searches := []Search{Web, Image, Video}
results := make([]Result, len(searches))
for i, search := range searches {
	i, search := i, search // https://golang.org/doc/faq#closures_and_goroutines
	g.Go(func() error {
		result, err := search(ctx, query)
		if err == nil {
			results[i] = result
		}
		return err
	})
}
if err := g.Wait(); err != nil {
	return nil, err
}
return results, nil

Hope this is helpful to those who are not aware of the errgroup package.

huangapple
  • 本文由 发表于 2017年5月24日 16:27:43
  • 转载请务必保留本文链接:https://go.coder-hub.com/44152988.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定