英文:
All Goroutines Are Asleep (The Go Programming Language)
问题
我正在阅读《The Go Programming Language》并学习goroutines,遇到了以下问题。在这个例子中,下面的函数旨在接收一个文件通道并处理其中的每个文件:
func makeThumbnails5(filenames <-chan string) int64 {
sizes := make(chan int64)
var wg sync.WaitGroup
for f := range filenames {
wg.Add(1)
// worker
go func(f string) {
defer wg.Done()
thumb, err := thumbnail.ImageFile(f)
if err != nil {
log.Println(err)
return
}
info, _ := os.Stat(thumb)
sizes <- info.Size()
}(f)
}
// closer
go func() {
wg.Wait()
close(sizes)
}()
var total int64
for size := range sizes {
total += size
}
wg.Wait()
return total
}
我尝试以以下方式使用这个函数:
func main() {
thumbnails := os.Args[1:] /* 从命令行获取所有图像的列表 */
ch := make(chan string, len(thumbnails))
for _, val := range thumbnails {
ch <- val
}
makeThumbnails5(ch)
}
然而,当我运行这个程序时,出现以下错误:
fatal error: all goroutines are asleep - deadlock!
似乎closer goroutine没有运行。有人可以帮我理解这里出了什么问题,以及如何正确运行这个函数吗?
英文:
I'm working through <a href="https://www.amazon.com/Programming-Language-Addison-Wesley-Professional-Computing/dp/0134190440">The Go Programming Language</a> and learning about goroutines, and came across the following issue. In this example, the following function is meant to take a channel of files and process each of them:
func makeThumbnails5(filenames <-chan string) int64 {
sizes := make(chan int64)
var wg sync.WaitGroup
for f := range filenames {
wg.Add(1)
// worker
go func(f string) {
defer wg.Done()
thumb, err := thumbnail.ImageFile(f)
if err != nil {
log.Println(err)
return
}
info, _ := os.Stat(thumb)
sizes <- info.Size()
}(f)
}
// closer
go func() {
wg.Wait()
close(sizes)
}()
var total int64
for size := range sizes {
total += size
}
wg.Wait()
return total
}
I've tried to use this function the following way:
func main() {
thumbnails := os.Args[1:] /* Get a list of all the images from the CLI */
ch := make(chan string, len(thumbnails))
for _, val := range thumbnails {
ch <- val
}
makeThumbnails5(ch)
}
However, when I run this program, I get the following error:
fatal error: all goroutines are asleep - deadlock!
It doesn't appear that the closer goroutine is running. Could someone help me understand what is going wrong here, and what I can do to run this function correctly?
答案1
得分: 1
我已经为您翻译了内容,请查看以下翻译结果:
正如我所评论的,它发生死锁是因为filenames
通道从未关闭,因此for f := range filenames
循环永远不会完成。然而,仅仅关闭输入通道意味着在循环中启动的所有goroutine都会在sizes <- info.Size()
这一行被阻塞,直到循环结束。在这种情况下没有问题,但如果输入很大,可能会出现问题(那么您可能还想限制并发工作者的数量)。因此,将主循环也放在一个goroutine中是有意义的,这样for size := range sizes
循环就可以开始消费了。以下代码应该可以工作:
func makeThumbnails5(filenames <-chan string) int64 {
sizes := make(chan int64)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for f := range filenames {
wg.Add(1)
// worker
go func(f string) {
defer wg.Done()
thumb, err := thumbnail.ImageFile(f)
if err != nil {
log.Println(err)
return
}
info, _ := os.Stat(thumb)
sizes <- info.Size()
}(f)
}
}()
// closer
go func() {
wg.Wait()
close(sizes)
}()
var total int64
for size := range sizes {
total += size
}
return total
}
主函数的实现也存在类似的问题,如果输入很大,您实际上是将其全部加载到内存(带缓冲的通道)中,然后再传递给进行处理。也许以下代码更好:
func main() {
ch := make(chan string)
go func(thumbnails []string) {
defer close(ch)
for _, val := range thumbnails {
ch <- val
}
}(os.Args[1:])
makeThumbnails5(ch)
}
英文:
As I commented it deadlocks because the filenames
chan is never closed and thus the for f := range filenames
loop never completes. However, just closing the input chan means that all goroutines launched in the loop would get stuck at the line sizes <- info.Size()
until the loop ends. Not a problem in this case but if the input can be huge it could be (then you'd probably want to limit the number of concurrent workers too). So it makes sense to have the main loop in a goroutine too so that the for size := range sizes
loop can start consuming. Following should work:
func makeThumbnails5(filenames <-chan string) int64 {
sizes := make(chan int64)
var wg sync.WaitGroup
wg.Add(1)
go func() {
defer wg.Done()
for f := range filenames {
wg.Add(1)
// worker
go func(f string) {
defer wg.Done()
thumb, err := thumbnail.ImageFile(f)
if err != nil {
log.Println(err)
return
}
info, _ := os.Stat(thumb)
sizes <- info.Size()
}(f)
}
}()
// closer
go func() {
wg.Wait()
close(sizes)
}()
var total int64
for size := range sizes {
total += size
}
return total
}
The implementation of the main has a similar problem that if the input is huge you're essentially load it all into memory (buffered chan) before passing it on to be processed. Perhaps something like following is better
func main() {
ch := make(chan string)
go func(thumbnails []string) {
defer close(ch)
for _, val := range thumbnails {
ch <- val
}
}(os.Args[1:])
makeThumbnails5(ch)
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论