当使用通道发送到多个切片时，可能会发生数据丢失。

huangapple
go评论113阅读模式
英文:

Data loss when sending to multiple slices using channel

问题

我最近在学习使用Go语言进行并发编程，我正在考虑编写一个程序，生成一系列的数字并将它们并发地发送到三个切片中。以下是代码：

  1. func main() {
  2.     ch := make(chan int)
  3.     done := make(chan bool)
  4.     var bag1 []int
  5.     var bag2 []int
  6.     var bag3 []int
  8.     go func() {
  9.         for i := 0; i < 1000000; i++ {
  10.             ch <- i
  11.         }
  12.         close(ch)
  13.         done <- true
  14.     }()
  16.     go sendToBag(&bag1, ch)
  17.     go sendToBag(&bag2, ch)
  18.     go sendToBag(&bag3, ch)
  20.     <-done
  22.     len1 := len(bag1)
  23.     len2 := len(bag2)
  24.     len3 := len(bag3)
  26.     fmt.Println("bag1的长度：", len1)
  27.     fmt.Println("bag2的长度：", len2)
  28.     fmt.Println("bag3的长度：", len3)
  29.     fmt.Println("总长度：", len1+len2+len3)
  30. }
  32. func sendToBag(bag *[]int, ch <-chan int) {
  33.     for n := range ch {
  34.         *bag = append(*bag, n)
  35.     }
  36. }

运行结果如下：

  1. bag1的长度：327643
  2. bag2的长度：335630
  3. bag3的长度：336725
  4. 总长度：999998

三个切片的总长度并不总是等于发送给它们的数字的数量。所以我的问题是：是什么导致了这个问题，如何改进代码？

英文:

I'm recently learning concurrency in golang, and I'm thinking a program with generate a series of number and then send them to three slices, concurrently. Here's the code:

  1. func main() {
  2. 	ch := make(chan int)
  3. 	done := make(chan bool)
  4. 	var bag1 []int
  5. 	var bag2 []int
  6. 	var bag3 []int
  8. 	go func() {
  9. 		for i := 0; i &lt; 1000000; i++ {
  10. 			ch &lt;- i
  11. 		}
  12. 		close(ch)
  13. 		done &lt;- true
  14. 	}()
  16. 	go sendToBag(&amp;bag1, ch)
  17. 	go sendToBag(&amp;bag2, ch)
  18. 	go sendToBag(&amp;bag3, ch)
  20. 	&lt;-done
  22. 	len1 := (len(bag1))
  23. 	len2 := (len(bag2))
  24. 	len3 := (len(bag3))
  26. 	fmt.Println(&quot;length of bag1:&quot;, len1)
  27. 	fmt.Println(&quot;length of bag2:&quot;, len2)
  28. 	fmt.Println(&quot;length of bag3:&quot;, len3)
  29. 	fmt.Println(&quot;total length:&quot;, len1+len2+len3)
  30. }
  32. func sendToBag(bag *[]int, ch &lt;-chan int) {
  33. 	for n := range ch {
  34. 		*bag = append(*bag, n)
  35. 	}
  36. }

which gives the output like the following:

  1. length of bag1: 327643
  2. length of bag2: 335630
  3. length of bag3: 336725
  4. total length: 999998

The total length of three slices doesn't always add up to the count of numbers which were send to them. So my question is: what caused the promblem and how to improve the code

答案1

得分: 5

你对done通道的使用并不能保证所有3个sendToBag goroutine完全完成它们的工作。

虽然在执行&lt;-done之前，for n := range ch {语句确实会将通道“完全排空”，但是你的代码中没有任何保证&lt;-done在最后的*bag = append(*bag, n)之后执行，因此不能保证len()调用仅在append()调用之后执行。

使用等待组（wait group）代替done通道。

  1. func main() {
  2. 	ch := make(chan int)
  3. 	wg := sync.WaitGroup{}
  4. 	var bag1 []int
  5. 	var bag2 []int
  6. 	var bag3 []int
  8. 	go func() {
  9. 		for i := 0; i < 1000000; i++ {
  10. 			ch <- i
  11. 		}
  12. 		close(ch)
  13. 	}()
  15. 	wg.Add(3)
  16. 	go sendToBag(&bag1, ch, &wg)
  17. 	go sendToBag(&bag2, ch, &wg)
  18. 	go sendToBag(&bag3, ch, &wg)
  20. 	wg.Wait()
  22. 	len1 := len(bag1)
  23. 	len2 := len(bag2)
  24. 	len3 := len(bag3)
  26. 	fmt.Println("bag1的长度:", len1)
  27. 	fmt.Println("bag2的长度:", len2)
  28. 	fmt.Println("bag3的长度:", len3)
  29. 	fmt.Println("总长度:", len1+len2+len3)
  30. }
  32. func sendToBag(bag *[]int, ch <-chan int, wg *sync.WaitGroup) {
  33. 	for n := range ch {
  34. 		*bag = append(*bag, n)
  35. 	}
  36. 	wg.Done()
  37. }

链接：https://go.dev/play/p/_wDgS5aS7bI

英文:

Your use of the done channel is simply not enough to guarantee that all 3 sendToBag goroutines finish their job completely.

While it's true that the channel is "fully drained" by the for n := range ch { statement before &lt;-done is executed, there is nothing in your code that ensures that &lt;-done is executed after the final *bag = append(*bag, n) is executed, and therefore there is no guarantee that the len() calls on the bags will be executed only after the append() calls.

Use a wait group instead of the done channel.

  1. func main() {
  2. 	ch := make(chan int)
  3. 	wg := sync.WaitGroup{}
  4. 	var bag1 []int
  5. 	var bag2 []int
  6. 	var bag3 []int
  8. 	go func() {
  9. 		for i := 0; i &lt; 1000000; i++ {
  10. 			ch &lt;- i
  11. 		}
  12. 		close(ch)
  13. 	}()
  15. 	wg.Add(3)
  16. 	go sendToBag(&amp;bag1, ch, &amp;wg)
  17. 	go sendToBag(&amp;bag2, ch, &amp;wg)
  18. 	go sendToBag(&amp;bag3, ch, &amp;wg)
  20. 	wg.Wait()
  22. 	len1 := (len(bag1))
  23. 	len2 := (len(bag2))
  24. 	len3 := (len(bag3))
  26. 	fmt.Println(&quot;length of bag1:&quot;, len1)
  27. 	fmt.Println(&quot;length of bag2:&quot;, len2)
  28. 	fmt.Println(&quot;length of bag3:&quot;, len3)
  29. 	fmt.Println(&quot;total length:&quot;, len1+len2+len3)
  30. }
  32. func sendToBag(bag *[]int, ch &lt;-chan int, wg *sync.WaitGroup) {
  33. 	for n := range ch {
  34. 		*bag = append(*bag, n)
  35. 	}
  36. 	wg.Done()
  37. }

https://go.dev/play/p/_wDgS5aS7bI

答案2

得分: 0

@mkopriva的答案是首选的，但如果你只想使用通道，这里是解决方案。

  1. func main() {
  2. 	ch := make(chan int)
  3. 	done := make(chan bool)
  5. 	var bag1 []int
  6. 	var bag2 []int
  7. 	var bag3 []int
  9. 	go func() {
  10. 		for i := 0; i < 1000000; i++ {
  11. 			ch <- i
  12. 		}
  13. 		close(ch)
  14. 		done <- true
  16. 	}()
  17. 	ch1 := make(chan bool)
  18. 	ch2 := make(chan bool)
  19. 	ch3 := make(chan bool)
  20. 	go sendToBag(&bag1, ch, ch1)
  21. 	go sendToBag(&bag2, ch, ch2)
  22. 	go sendToBag(&bag3, ch, ch3)
  23. 	<-ch1
  24. 	<-ch2
  25. 	<-ch3
  26. 	<-done
  28. 	len1 := len(bag1)
  29. 	len2 := len(bag2)
  30. 	len3 := len(bag3)
  32. 	fmt.Println("bag1的长度:", len1)
  33. 	fmt.Println("bag2的长度:", len2)
  34. 	fmt.Println("bag3的长度:", len3)
  35. 	fmt.Println("总长度:", len1+len2+len3)
  36. }
  38. func sendToBag(bag *[]int, ch <-chan int, ch1 chan bool) {
  40. 	for n := range ch {
  41. 		*bag = append(*bag, n)
  42. 	}
  43. 	ch1 <- true
  45. }

在sendToBag函数中添加了更多的通道，因此主goroutine将被阻塞，直到其他goroutine在ch1/ch2/ch3上写入数据，这样可以确保在执行最后的*bag = append(*bag, n)之后执行<-done。

英文:

@mkopriva's answer is preferred but if you want to do with channel only, here is the solution.

  1. func main() {
  2. 	ch := make(chan int)
  3. 	done := make(chan bool)
  5. 	var bag1 []int
  6. 	var bag2 []int
  7. 	var bag3 []int
  9. 	go func() {
  10. 		for i := 0; i &lt; 1000000; i++ {
  11. 			ch &lt;- i
  12. 		}
  13. 		close(ch)
  14. 		done &lt;- true
  16. 	}()
  17. 	ch1 := make(chan bool)
  18. 	ch2 := make(chan bool)
  19. 	ch3 := make(chan bool)
  20. 	go sendToBag(&amp;bag1, ch, ch1)
  21. 	go sendToBag(&amp;bag2, ch, ch2)
  22. 	go sendToBag(&amp;bag3, ch, ch3)
  23. 	&lt;-ch1
  24. 	&lt;-ch2
  25. 	&lt;-ch3
  26. 	&lt;-done
  28. 	len1 := (len(bag1))
  29. 	len2 := (len(bag2))
  30. 	len3 := (len(bag3))
  32. 	fmt.Println(&quot;length of bag1:&quot;, len1)
  33. 	fmt.Println(&quot;length of bag2:&quot;, len2)
  34. 	fmt.Println(&quot;length of bag3:&quot;, len3)
  35. 	fmt.Println(&quot;total length:&quot;, len1+len2+len3)
  36. }
  38. func sendToBag(bag *[]int, ch &lt;-chan int, ch1 chan bool) {
  40. 	for n := range ch {
  41. 		*bag = append(*bag, n)
  42. 	}
  43. 	ch1 &lt;- true
  45. }

https://go.dev/play/p/id3l_VtUq8E

Here we add more channels in the sendToBag function, so the main goroutine will be blocked until some other goroutine write on ch1/ch2/ch3, which ensures <-done is executed after the final *bag = append(*bag, n) is executed.

huangapple
  • 本文由 发表于 2022年2月27日 19:22:36
  • 转载请务必保留本文链接：https://go.coder-hub.com/71284340.html
  • concurrency
  • go
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定