为什么这个 Go 程序中存在数据竞争?

huangapple go评论84阅读模式
英文:

Why is there a data race in this Go program?

问题

我正在尝试将日志消息存储在缓冲区中,只有在出现错误时才能访问它们,有点像更智能的日志处理,机会主义日志记录的情况。在这个例子中,我每5秒从缓冲区获取日志,但是当我使用go run -race code.go运行它时,会出现数据竞争。

我正在使用通道进行通信,但显然做错了什么。

package main

import (
	"bytes"
	"fmt"
	"io/ioutil"
	"log"
	"time"
)

type LogRequest struct {
	Buffer chan []byte
}

type LogBuffer struct {
	LogInputChan chan []byte
	LogRequests  chan LogRequest
}

func (f LogBuffer) Write(b []byte) (n int, err error) {
	f.LogInputChan <- b
	return len(b), nil
}

func main() {
	var logBuffer LogBuffer
	logBuffer.LogInputChan = make(chan []byte, 100)
	logBuffer.LogRequests = make(chan LogRequest, 100)

	log.SetOutput(logBuffer)

	// store the log messages in a buffer until we ask for it
	go func() {
		buf := new(bytes.Buffer)

		for {
			select {
			// receive log messages
			case logMessage := <-logBuffer.LogInputChan:
				buf.Write(logMessage) // <- data race
			case logRequest := <-logBuffer.LogRequests:
				c, errReadAll := ioutil.ReadAll(buf)
				if errReadAll != nil {
					panic(errReadAll)
				}
				logRequest.Buffer <- c
			}
		}
	}()

	// log a test message every 1 second
	go func() {
		for i := 0; i < 30; i++ {
			log.Printf("test: %d", i) // <- data race
			time.Sleep(1 * time.Second)
		}
	}()

	// print the log every 5 seconds
	go func() {
		for {
			time.Sleep(5 * time.Second)

			var logRequest LogRequest
			logRequest.Buffer = make(chan []byte, 1)
			logBuffer.LogRequests <- logRequest

			buffer := <-logRequest.Buffer

			fmt.Printf("**** LOG *****\n%s**** END *****\n\n", buffer)
		}
	}()

	time.Sleep(45 * time.Second)
}
英文:

I'm trying to store log messages in a buffer to access them only when I get an error. A bit like in Smarter log handling, the case for opportunistic logging. In this example I fetch the logs from the buffer each 5 seconds but I get a data race when I run it with go run -race code.go.

I'm using channels to communicate but I'm doing something wrong, obviously.

<!-- language: lang-golang -->

package main
import (
&quot;bytes&quot;
&quot;fmt&quot;
&quot;io/ioutil&quot;
&quot;log&quot;
&quot;time&quot;
)
type LogRequest struct {
Buffer chan []byte
}
type LogBuffer struct {
LogInputChan chan []byte
LogRequests  chan LogRequest
}
func (f LogBuffer) Write(b []byte) (n int, err error) {
f.LogInputChan &lt;- b
return len(b), nil
}
func main() {
var logBuffer LogBuffer
logBuffer.LogInputChan = make(chan []byte, 100)
logBuffer.LogRequests = make(chan LogRequest, 100)
log.SetOutput(logBuffer)
// store the log messages in a buffer until we ask for it
go func() {
buf := new(bytes.Buffer)
for {
select {
// receive log messages
case logMessage := &lt;-logBuffer.LogInputChan:
buf.Write(logMessage) // &lt;- data race
case logRequest := &lt;-logBuffer.LogRequests:
c, errReadAll := ioutil.ReadAll(buf)
if errReadAll != nil {
panic(errReadAll)
}
logRequest.Buffer &lt;- c
}
}
}()
// log a test message every 1 second
go func() {
for i := 0; i &lt; 30; i++ {
log.Printf(&quot;test: %d&quot;, i) // &lt;- data race
time.Sleep(1 * time.Second)
}
}()
// print the log every 5 seconds
go func() {
for {
time.Sleep(5 * time.Second)
var logRequest LogRequest
logRequest.Buffer = make(chan []byte, 1)
logBuffer.LogRequests &lt;- logRequest
buffer := &lt;-logRequest.Buffer
fmt.Printf(&quot;**** LOG *****\n%s**** END *****\n\n&quot;, buffer)
}
}()
time.Sleep(45 * time.Second)
}

答案1

得分: 8

log包使用一个内部缓冲区来构建日志消息以进行输出(在log/Logger中的buf字段)。它组合了头部信息,附加了调用者提供的数据,然后将该缓冲区传递给你的Write方法进行输出。

为了减少内存分配,log包对每条日志消息都会重用这个缓冲区。虽然文档中没有明确说明,但隐含的假设是你的Write方法在Write调用期间只使用提供的[]byte数据。对于大多数输出(如文件或标准输出),这个假设是可以的。

为了避免数据竞争,你需要在从Write函数返回之前对传入的数据进行显式拷贝:

func (f LogBuffer) Write(b []byte) (n int, err error) {
    z := make([]byte, len(b))
    copy(z, b)
    f.LogInputChan <- z
    return len(b), nil
}
英文:

The log package uses an internal buffer to build-up log messages for output (the buf field in log/Logger). It composes the header, appends the data provided by the caller, then passes this buffer to your Write method for output.

In order to reduce allocations, the log package recycles this buffer for each log message. It's not stated in the documentation, but the implicit assumption is that your Write method only uses the provided []byte data for the duration of the Write call. This assumption is OK for most outputs, e.g. a file or STDOUT.

To avoid a data race, you need to make an explicit copy of the incoming data before returning from the Write function:

func (f LogBuffer) Write(b []byte) (n int, err error) {
z := make([]byte, len(b))
copy(z, b)
f.LogInputChan &lt;- z
return len(b), nil
}

huangapple
  • 本文由 发表于 2013年12月20日 01:06:48
  • 转载请务必保留本文链接:https://go.coder-hub.com/20687597.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定