pass message by sending buffer to go channel but got overwritten, why?

A very simple and usual case in golang as below, but got result not expected.

package main

import (
	"fmt"
	"time"
)

func main() {
	consumer(generator())
	for {
		time.Sleep(time.Duration(time.Second))
	}
}

// simple generator through channel
func generator() <-chan []byte {
	ret := make(chan []byte)
	go func() {
		// make buf outside of loop, and result is not expected
		var ch = byte('A')
		count := 0
		buf := make([]byte, 1)
		for {
			if count > 10 {
				return
			}
			// make buf inside loop, and result is expected
			// buf := make([]byte, 1)
			buf[0] = ch
			ret <- buf
			ch++
			count++
			// time.Sleep(time.Duration(time.Second))
		}
	}()
	return ret
}

// simple consumer through channel
func consumer(recv <-chan []byte) {
	go func() {
		for buf := range recv {
			fmt.Println("received:" + string(buf[0]))
		}
	}()
}

output:
received:A
received:B
received:D
received:D
received:F
received:F
received:H
received:H
received:J
received:J
received:K

In generator, if put the buf variable inside for loop, result is what I expected:

received:A
received:B
received:C
received:D
received:E
received:F
received:G
received:H
received:I
received:J
received:K

I am thinking even buf is outside for loop and not changed always, after we write it to channe, receiver will read out it until next write can happen, so its' content should not be override, but looks like golang behaviors not in this way, what wrong for happened here?

Problem: your code contains a data race

Save your your program in a file named main.go; then run it with the race detector: go run -race main.go. You should see something like the following:

$ go run -race main.go 
received:A
==================
WARNING: DATA RACE
Write at 0x00c000180000 by goroutine 7:
  main.generator.func1()
      /redacted/main.go:29 +0x8c

Previous read at 0x00c000180000 by goroutine 8:
  main.consumer.func1()
      /redacted/main.go:43 +0x55

The race detector tells you your program contains a data race because two goroutines are writing and reading to some shared memory without synchronisation:

• the anonymous function launched as a goroutine in your generator function updates its local variable named buf at line 29;
• the anonymous function launched as a goroutine in your consumer function reads from its local variable named buf at line 43.

The data race stems from the conjunction of two things:

1. Although local variable buf in consumer is just a copy of the homonymous local variable in generator, those slice variables are coupled because they refer to the same underlying array.

   

   See [the relevant section of the language specification] (https://golang.org/ref/spec#Slice_types):

   > A slice, once initialized, is always associated with an underlying array that holds its elements. A slice therefore shares storage with its array and with other slices of the same array [...]

2. Operations on slices are not concurrency-safe and require proper synchronisation if performed concurrently (i.e. from multiple goroutines at the same time).

What your code displays is a typical case of aliasing. You should better familiarise yourself with how slices work.

Solution

You could eliminate the data race by using a one-byte array ([1]byte) instead of a slice, but arrays are quite inflexible in Go. Whether you really need to use a slice of bytes at all here is unclear. Since you're effectively only sending one byte at a time to the channel, why not simply use a chan byte rather than a chan []byte?

Other improvements unrelated to the data race include:

• modifying the API of your two functions to make them synchronous (and therefore, easier to reason about);

• simplifying the generator logic and closing the channel so that main can actually terminate;

• simplifying the consumer logic and not spawning a goroutine for it.

  package main
  
  import "fmt"
  
  func main() {
      ch := make(chan byte)
      go generator(ch)
      consumer(ch)
  }
  
  func generator(ch chan<- byte) {
      var c byte = 'A'
      for i := 0; i < 10; i++ {
          ch <- c
  	      c++
      }
      close(ch)
  }
  
  func consumer(ch <-chan byte) {
      for c := range ch {
          fmt.Printf("received: %c\n", c)
      }
  }
  

The case is very simple. Both threads have ownership of the buffer and so channel does not guarantee synchronization. While consumer is reading the channel, generator is fast enough to modify the buffer so this char skip happens. to fix this you have to introduce another channel (that will send buffer back) or pass a copy of buffer.