英文:
Why is there a data race in this Go program?
问题
我正在尝试将日志消息存储在缓冲区中,只有在出现错误时才能访问它们,有点像更智能的日志处理,机会主义日志记录的情况。在这个例子中,我每5秒从缓冲区获取日志,但是当我使用go run -race code.go
运行它时,会出现数据竞争。
我正在使用通道进行通信,但显然做错了什么。
package main
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"time"
)
type LogRequest struct {
Buffer chan []byte
}
type LogBuffer struct {
LogInputChan chan []byte
LogRequests chan LogRequest
}
func (f LogBuffer) Write(b []byte) (n int, err error) {
f.LogInputChan <- b
return len(b), nil
}
func main() {
var logBuffer LogBuffer
logBuffer.LogInputChan = make(chan []byte, 100)
logBuffer.LogRequests = make(chan LogRequest, 100)
log.SetOutput(logBuffer)
// store the log messages in a buffer until we ask for it
go func() {
buf := new(bytes.Buffer)
for {
select {
// receive log messages
case logMessage := <-logBuffer.LogInputChan:
buf.Write(logMessage) // <- data race
case logRequest := <-logBuffer.LogRequests:
c, errReadAll := ioutil.ReadAll(buf)
if errReadAll != nil {
panic(errReadAll)
}
logRequest.Buffer <- c
}
}
}()
// log a test message every 1 second
go func() {
for i := 0; i < 30; i++ {
log.Printf("test: %d", i) // <- data race
time.Sleep(1 * time.Second)
}
}()
// print the log every 5 seconds
go func() {
for {
time.Sleep(5 * time.Second)
var logRequest LogRequest
logRequest.Buffer = make(chan []byte, 1)
logBuffer.LogRequests <- logRequest
buffer := <-logRequest.Buffer
fmt.Printf("**** LOG *****\n%s**** END *****\n\n", buffer)
}
}()
time.Sleep(45 * time.Second)
}
英文:
I'm trying to store log messages in a buffer to access them only when I get an error. A bit like in Smarter log handling, the case for opportunistic logging. In this example I fetch the logs from the buffer each 5 seconds but I get a data race when I run it with go run -race code.go
.
I'm using channels to communicate but I'm doing something wrong, obviously.
<!-- language: lang-golang -->
package main
import (
"bytes"
"fmt"
"io/ioutil"
"log"
"time"
)
type LogRequest struct {
Buffer chan []byte
}
type LogBuffer struct {
LogInputChan chan []byte
LogRequests chan LogRequest
}
func (f LogBuffer) Write(b []byte) (n int, err error) {
f.LogInputChan <- b
return len(b), nil
}
func main() {
var logBuffer LogBuffer
logBuffer.LogInputChan = make(chan []byte, 100)
logBuffer.LogRequests = make(chan LogRequest, 100)
log.SetOutput(logBuffer)
// store the log messages in a buffer until we ask for it
go func() {
buf := new(bytes.Buffer)
for {
select {
// receive log messages
case logMessage := <-logBuffer.LogInputChan:
buf.Write(logMessage) // <- data race
case logRequest := <-logBuffer.LogRequests:
c, errReadAll := ioutil.ReadAll(buf)
if errReadAll != nil {
panic(errReadAll)
}
logRequest.Buffer <- c
}
}
}()
// log a test message every 1 second
go func() {
for i := 0; i < 30; i++ {
log.Printf("test: %d", i) // <- data race
time.Sleep(1 * time.Second)
}
}()
// print the log every 5 seconds
go func() {
for {
time.Sleep(5 * time.Second)
var logRequest LogRequest
logRequest.Buffer = make(chan []byte, 1)
logBuffer.LogRequests <- logRequest
buffer := <-logRequest.Buffer
fmt.Printf("**** LOG *****\n%s**** END *****\n\n", buffer)
}
}()
time.Sleep(45 * time.Second)
}
答案1
得分: 8
log
包使用一个内部缓冲区来构建日志消息以进行输出(在log/Logger中的buf
字段)。它组合了头部信息,附加了调用者提供的数据,然后将该缓冲区传递给你的Write
方法进行输出。
为了减少内存分配,log
包对每条日志消息都会重用这个缓冲区。虽然文档中没有明确说明,但隐含的假设是你的Write
方法在Write
调用期间只使用提供的[]byte
数据。对于大多数输出(如文件或标准输出),这个假设是可以的。
为了避免数据竞争,你需要在从Write
函数返回之前对传入的数据进行显式拷贝:
func (f LogBuffer) Write(b []byte) (n int, err error) {
z := make([]byte, len(b))
copy(z, b)
f.LogInputChan <- z
return len(b), nil
}
英文:
The log
package uses an internal buffer to build-up log messages for output (the buf
field in log/Logger). It composes the header, appends the data provided by the caller, then passes this buffer to your Write
method for output.
In order to reduce allocations, the log
package recycles this buffer for each log message. It's not stated in the documentation, but the implicit assumption is that your Write
method only uses the provided []byte
data for the duration of the Write
call. This assumption is OK for most outputs, e.g. a file or STDOUT.
To avoid a data race, you need to make an explicit copy of the incoming data before returning from the Write
function:
func (f LogBuffer) Write(b []byte) (n int, err error) {
z := make([]byte, len(b))
copy(z, b)
f.LogInputChan <- z
return len(b), nil
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论