英文:
Concurrent limited consumers using goroutins and channels
问题
我尝试从《Effective Go》中复现了“管理资源的一种有效方法是启动固定数量的处理goroutine，它们都从请求通道中读取。”这句话，并发现了一个致命错误：所有的goroutine都处于休眠状态 - 死锁！
思路很简单：有一个队列和一个结果通道，以及若干个有限数量的“工作者”。
我的代码在Go Playground中：
queue := make(chan *Request)
result := make(chan int)
quit := make(chan bool)
go Serve(queue, quit)
for i := 0; i < 10; i++ {
req := Request{i, result}
queue <- &req
}
close(queue)
for i := 0; i < 10; i++ {
fmt.Printf("Finished %d\n", <-result)
}
fmt.Printf("All finished\n")
quit <- true
Serve函数：
func handle(queue chan *Request) {
for r := range queue {
//process(r)
fmt.Printf("Processing %d\n", r.i)
r.r <- r.i
}
}
func Serve(clientRequests chan *Request, quit chan bool) {
MaxOutstanding := 2
// 启动处理器
for i := 0; i < MaxOutstanding; i++ {
go handle(clientRequests)
}
<-quit // 等待退出信号
}
有什么问题吗？或者是否有更简单的解决方案来实现处理请求的有限数量的工作者？
英文:
I've tried to reproduce "approach that manages resources well is to start a fixed number of handle goroutines all reading from the request channel." from Effective Go and found
fatal error: all goroutines are asleep - deadlock!
The idea is simple: have 1 queue and 1 results channels and several limited number of "workers".
My code is in Go Playground
queue := make(chan *Request)
result := make(chan int)
quit := make(chan bool)
go Serve(queue, quit)
for i := 0; i < 10; i++ {
req := Request{i, result}
queue <- &req
}
close(queue)
for i := 0; i < 10; i++ {
fmt.Printf("Finished %d\n", <-result)
}
fmt.Printf("All finished\n")
quit <- true
Function Serve:
func handle(queue chan *Request) {
for r := range queue {
//process(r)
fmt.Printf("Processing %d\n", r.i)
r.r <- r.i
}
}
func Serve(clientRequests chan *Request, quit chan bool) {
MaxOutstanding := 2
// Start handlers
for i := 0; i < MaxOutstanding; i++ {
go handle(clientRequests)
}
<-quit // Wait to be told to exit.
}
What is wrong? Or may be there is simplier solution for implementing limited number of workers that process requests?
答案1
得分: 1
结果通道没有缓冲区，所以每次发送都会阻塞 goroutine，直到有接收者接收到结果。因此，在处理完前两个项目后，两个处理程序例程正在等待接收它们的结果。但是，接收工作项的代码尚未运行，所以它们被阻塞了。然后，main() 尝试发送下一个工作项，而处理程序还没有准备好接收它，所以 main() 现在也被阻塞了。
修复的一种方法是在将工作项排队之前启动一个 goroutine 在后台接收结果。这是你的代码的一个版本，它在后台接收结果，并使 main() 等待直到所有结果都被接收和处理：http://play.golang.org/p/_CKn3CxQFc。
在你的示例中，你可以通过计算接收到的结果数量来知道何时安全退出。但是，如果出现需要确定何时完成且计数不足以解决的情况，那么可以采取以下步骤：1）每个工作程序可以在完成时发出信号，2）Serve 可以在所有工作程序发出完成信号后关闭结果通道，3）当所有结果都被处理时，结果处理的 goroutine 可以向 main() 发送一个 quit。有很多变种；你可以在这里看到代码：http://play.golang.org/p/12wZbm0rxa
英文:
The result channel isn't buffered, so each send blocks the goroutine until something receives the result. So after the first two items are processed, the two handler routines are waiting for their results to be received. But the code to receive work items isn't running yet, so they're stuck. Then main() tries to send the next work item, and the handlers aren't ready to receive it so main() is now stuck too.
One way to fix it is to start a goroutine to receive results in the background before you queue the work items. Here's a version of your code that receives the results in the background, plus makes main() wait until all results have been received and handled: http://play.golang.org/p/_CKn3CxQFc.
In your example, you can know when it's safe to quit just by counting results received. But if a situation comes up where you need to figure out when you're done and counting is not enough, then 1) each worker can signal when it's done, 2) Serve can close the result channel once all workers signal done, 3) your result-handling goroutine can send a quit to main() when all the results have been handled. There are lots of variations on that; you can see code at http://play.golang.org/p/12wZbm0rxa
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论