英文:
concurrently changing a file offset using `file.Seek` escapes the race detector?
问题
在下面的代码中，竞争检测器没有检测到对file.Seek的不当调用，这与我根据该方法的签名和其实现（假设为Unix）所期望的不符。
你能帮忙解释一下内部发生了什么吗？
func main() {
file, _ := os.Open("main.go")
st, _ := file.Stat()
max := st.Size()
limit := int64(25)
start := make(chan bool)
type res struct {
bytes.Buffer
index int64
}
out := make(chan res)
var wg sync.WaitGroup
for i := int64(0); i*limit < max; i++ {
wg.Add(1)
go func(i, limit int64) {
<-start
file.Seek(i*limit, 0) // 这一行应该会触发竞争，为什么它能逃过竞争检测器？
r := res{index: i}
io.Copy(&r, io.LimitReader(file, limit))
out <- r
wg.Done()
}(i, limit)
}
close(start)
go func() {
wg.Wait()
close(out)
}()
var last int64
buf := []res{}
for o := range out {
buf = append(buf, o)
sort.Slice(buf, func(i int, j int) bool {
return buf[i].index < buf[j].index
})
for len(buf) > 0 && buf[0].index == last {
fmt.Fprintf(os.Stdout, "%v\n", buf[0].String())
// fmt.Fprintf(os.Stdout, "%v\n", buf[0].index)
buf = append(buf[:0], buf[1:]...)
last++
}
}
sort.Slice(buf, func(i int, j int) bool {
return buf[i].index < buf[j].index
})
for len(buf) > 0 {
fmt.Fprintf(os.Stdout, "%v\n", buf[0].String())
// fmt.Fprintf(os.Stdout, "%v\n", buf[0].index)
buf = append(buf[:0], buf[1:]...)
last++
}
log.Println(last)
}
我使用以下命令运行它：
$ while go run -race .; do :; done
ok
ok
ok
ok
...
但没有运气。
英文:
in below code, the race detector does not detect the racy call to file.Seek as i expected reading the signature of the method and its body (assuming unix).
Can you help explaining what is going on under the hood ?
func main() {
file, _ := os.Open("main.go")
st, _ := file.Stat()
max := st.Size()
limit := int64(25)
start := make(chan bool)
type res struct {
bytes.Buffer
index int64
}
out := make(chan res)
var wg sync.WaitGroup
for i := int64(0); i*limit < max; i++ {
wg.Add(1)
go func(i, limit int64) {
<-start
file.Seek(i*limit, 0) // This line should trigger race ? How does it work under the hood so it escapes the race detector ?
r := res{index: i}
io.Copy(&r, io.LimitReader(file, limit))
out <- r
wg.Done()
}(i, limit)
}
close(start)
go func() {
wg.Wait()
close(out)
}()
var last int64
buf := []res{}
for o := range out {
buf = append(buf, o)
sort.Slice(buf, func(i int, j int) bool {
return buf[i].index < buf[j].index
})
for len(buf) > 0 && buf[0].index == last {
fmt.Fprintf(os.Stdout, "%v\n", buf[0].String())
// fmt.Fprintf(os.Stdout, "%v\n", buf[0].index)
buf = append(buf[:0], buf[1:]...)
last++
}
}
sort.Slice(buf, func(i int, j int) bool {
return buf[i].index < buf[j].index
})
for len(buf) > 0 {
fmt.Fprintf(os.Stdout, "%v\n", buf[0].String())
// fmt.Fprintf(os.Stdout, "%v\n", buf[0].index)
buf = append(buf[:0], buf[1:]...)
last++
}
log.Println(last)
}
I run it with
$ while go run -race .; do :; done
ok
ok
ok
ok
...
but no luck.
答案1
得分: 4
这里确实存在一场竞争，但这场竞争发生在Go生态系统之外。因此，Go生态系统中的竞争检测器无法察觉到这场竞争并进行检测。
需要注意的是，一些操作系统具有pread和pwrite系统调用，这些系统调用接受文件位置参数，因此它们可以执行原子读取/写入特定位置的操作，而不是使用两个分离的系统调用（例如lseek + read）。如果您的系统具有这些调用，并且还具有readv和writev调用，那么它可能也具有preadv和pwritev调用。
请参阅https://pkg.go.dev/syscall#Pread和https://pkg.go.dev/syscall#Pwrite。
注：在（未来的）某些系统上，竞争可能发生在Go生态系统内部，这种情况下，竞争检测器可能会检测到它。不过，我不会对此抱有太大期望。
英文:
There is a race here, but the race happens outside the Go ecosystem.<sup>1</sup> As such, the race detector—which is part of the Go ecosystem—never sees the race and doesn't detect it.
Note that some operating systems have pread and pwrite system calls that take a file-position argument, so that they can do atomic read/write-at-particular-position operations, rather than using two separate (and therefore race-enabling) system calls (lseek + read for instance). If your system has these calls and also has the readv and writev calls, it probably has preadv and pwritev as well.
See also https://pkg.go.dev/syscall#Pread and https://pkg.go.dev/syscall#Pwrite.
<sup>1</sup>There could be (future) systems on which the race happens inside the Go ecosystem, in which case, the race detector might detect it. I wouldn't count on it though.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论