英文:
Testing go channel throughput - all goroutines deadlock
问题
我写了一个小程序来测试Go通道的吞吐量，但它总是发生死锁，我尝试了很多努力，但无法理解原因：
package main
import (
"fmt"
"runtime"
)
const CONCURRENCY = 32
const WORK_PER_WORKER = 100
const TOTAL_WORK = CONCURRENCY * WORK_PER_WORKER
func work() {
sum := 0
for i := 0; i < 10000000; i++ {
sum *= i
}
}
type WorkItem struct {
Done chan int
}
func main() {
runtime.GOMAXPROCS(CONCURRENCY)
var workQueue [CONCURRENCY]chan *WorkItem
// 初始化工作队列
for i := 0; i < CONCURRENCY; i++ {
workQueue[i] = make(chan *WorkItem)
}
// 启动工作协程
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
anItem := <-workQueue[i]
work()
anItem.Done <- 1
}(i)
}
completed := make(chan bool, TOTAL_WORK)
for i := 0; i < TOTAL_WORK; i++ {
go func(i int) {
// 发送工作到队列
workToDo := &WorkItem{Done: make(chan int)}
workQueue[i/WORK_PER_WORKER] <- workToDo // !! 死锁
// 等待工作完成
<-workToDo.Done
completed <- true
}(i)
}
fmt.Println("等待中")
for i := 0; i < TOTAL_WORK; i++ {
<-completed
}
}
英文:
I made a small program to benchmark go channel throughput, however it always deadlocks, I tried very hard but cannot understand why:
package main
import (
"fmt"
"runtime"
)
const CONCURRENCY = 32
const WORK_PER_WORKER = 100
const TOTAL_WORK = CONCURRENCY * WORK_PER_WORKER
func work() {
sum := 0
for i := 0; i < 10000000; i++ {
sum *= i
}
}
type WorkItem struct {
Done chan int
}
func main() {
runtime.GOMAXPROCS(CONCURRENCY)
var workQueue [CONCURRENCY]chan *WorkItem
// initialize workers
for i := 0; i < CONCURRENCY; i++ {
workQueue[i] = make(chan *WorkItem)
}
// start workers
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
anItem := <-workQueue[i]
work()
anItem.Done <- 1
}(i)
}
completed := make(chan bool, TOTAL_WORK)
for i := 0; i < TOTAL_WORK; i++ {
go func(i int) {
// send work to queues
workToDo := &WorkItem{Done: make(chan int)}
workQueue[i/WORK_PER_WORKER] <- workToDo // !! DEADLOCK
// wait until the work is done
<-workToDo.Done
completed <- true
}(i)
}
fmt.Println("Waiting")
for i := 0; i < TOTAL_WORK; i++ {
<-completed
}
}
答案1
得分: 2
你的代码go func(i int) { anItem := <-workQueue[i]; ... }只从workQueue[i]中移除了一个项目，但你试图将WORK_PER_WORKER个项目放入其中。你将处理CONCURRENCY个项目，之后所有的读取goroutine都已终止，你就会遇到死锁。
在worker goroutine中循环执行可以“解决”你的死锁问题：http://play.golang.org/p/j2pavqnBDv
只是“解决”了，因为这些worker goroutine永远不会终止。也许你可以尝试使用close关闭通道，以通知worker goroutine在不发送任何内容时终止。
英文:
Your code go func(i int) { anItem := <-workQueue[i]; ... } removes juste 1 item from workQueue[i] but you are trying to stuff WORK_PER_WORKER items into it. You will work on CONCURRENCY many items and after that all reading goroutines have terminated and you have your deadlock.
Looping in the worker goroutines "solves" your deadlock: http://play.golang.org/p/j2pavqnBDv
Just "solves" because these worker goroutines will never terminate. Maybe you can experiment with closeing your channels to notify the worker goroutines when nothing will be sent.
答案2
得分: 1
因为你的工作程序只处理一个任务然后退出。因此，只有前CONCURRENCY个项目会被处理，然后workQueue[i/WORK_PER_WORKER] <- workToDo会无限期地阻塞。因此，completed通道永远无法接收到足够的值，main也会永远阻塞。
你的工作程序应该使用循环来执行任务，像这样：
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
for anItem := range workQueue[i] {
work()
anItem.Done <- 1
}
}(i)
}
英文:
Because your worker only process one task and then exit. Thus, only first CONCURRENCY items proceed and then workQueue[i/WORK_PER_WORKER] <- workToDo blocks indifinitely. Thus, completed chan never receive enough values and main also blocks forever.
Your worker should do work in loops, like this:
for i := 0; i < CONCURRENCY; i++ {
go func(i int) {
for anItem := range workQueue[i] {
work()
anItem.Done <- 1
}
}(i)
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论