《Go语言圣经》中的一句话是“所有的Goroutine都在休眠”。

huangapple go评论85阅读模式
英文:

All Goroutines Are Asleep (The Go Programming Language)

问题

我正在阅读《The Go Programming Language》并学习goroutines,遇到了以下问题。在这个例子中,下面的函数旨在接收一个文件通道并处理其中的每个文件:

func makeThumbnails5(filenames <-chan string) int64 {
    sizes := make(chan int64)
    var wg sync.WaitGroup
    for f := range filenames {
        wg.Add(1)
        // worker
        go func(f string) {
            defer wg.Done()
            thumb, err := thumbnail.ImageFile(f)
            if err != nil {
                log.Println(err)
                return
            }
            info, _ := os.Stat(thumb)
            sizes <- info.Size()
        }(f)
    }

    // closer
    go func() {
        wg.Wait()
        close(sizes)
    }()

    var total int64
    for size := range sizes {
        total += size
    }

    wg.Wait()
    return total
}

我尝试以以下方式使用这个函数:

func main() {
    thumbnails := os.Args[1:] /* 从命令行获取所有图像的列表 */
    ch := make(chan string, len(thumbnails))
    for _, val := range thumbnails {
        ch <- val
    }
    makeThumbnails5(ch)
}

然而,当我运行这个程序时,出现以下错误:

fatal error: all goroutines are asleep - deadlock!

似乎closer goroutine没有运行。有人可以帮我理解这里出了什么问题,以及如何正确运行这个函数吗?

英文:

I'm working through <a href="https://www.amazon.com/Programming-Language-Addison-Wesley-Professional-Computing/dp/0134190440">The Go Programming Language</a> and learning about goroutines, and came across the following issue. In this example, the following function is meant to take a channel of files and process each of them:

func makeThumbnails5(filenames &lt;-chan string) int64 {
	sizes := make(chan int64)
	var wg sync.WaitGroup
	for f := range filenames {
		wg.Add(1)
		// worker
		go func(f string) {
			defer wg.Done()
			thumb, err := thumbnail.ImageFile(f)
			if err != nil {
				log.Println(err)
				return
			}
			info, _ := os.Stat(thumb)
			sizes &lt;- info.Size()
		}(f)
	}

	// closer
	go func() {
		wg.Wait()
		close(sizes)
	}()

	var total int64
	for size := range sizes {
		total += size
	}

	wg.Wait()
	return total
}

I've tried to use this function the following way:

func main() {
	thumbnails := os.Args[1:] /* Get a list of all the images from the CLI */
	ch := make(chan string, len(thumbnails))
	for _, val := range thumbnails {
		ch &lt;- val
	}
	makeThumbnails5(ch)
}

However, when I run this program, I get the following error:

fatal error: all goroutines are asleep - deadlock!

It doesn't appear that the closer goroutine is running. Could someone help me understand what is going wrong here, and what I can do to run this function correctly?

答案1

得分: 1

我已经为您翻译了内容,请查看以下翻译结果:

正如我所评论的,它发生死锁是因为filenames通道从未关闭,因此for f := range filenames循环永远不会完成。然而,仅仅关闭输入通道意味着在循环中启动的所有goroutine都会在sizes <- info.Size()这一行被阻塞,直到循环结束。在这种情况下没有问题,但如果输入很大,可能会出现问题(那么您可能还想限制并发工作者的数量)。因此,将主循环也放在一个goroutine中是有意义的,这样for size := range sizes循环就可以开始消费了。以下代码应该可以工作:

func makeThumbnails5(filenames <-chan string) int64 {
    sizes := make(chan int64)
    var wg sync.WaitGroup
    wg.Add(1)
    go func() {
        defer wg.Done()
        for f := range filenames {
            wg.Add(1)
            // worker
            go func(f string) {
                defer wg.Done()
                thumb, err := thumbnail.ImageFile(f)
                if err != nil {
                    log.Println(err)
                    return
                }
                info, _ := os.Stat(thumb)
                sizes <- info.Size()
            }(f)
        }
    }()

    // closer
    go func() {
        wg.Wait()
        close(sizes)
    }()

    var total int64
    for size := range sizes {
        total += size
    }

    return total
}

主函数的实现也存在类似的问题,如果输入很大,您实际上是将其全部加载到内存(带缓冲的通道)中,然后再传递给进行处理。也许以下代码更好:

func main() {
    ch := make(chan string)
    go func(thumbnails []string) {
        defer close(ch)
        for _, val := range thumbnails {
            ch <- val
        }
    }(os.Args[1:])
    makeThumbnails5(ch)
}
英文:

As I commented it deadlocks because the filenames chan is never closed and thus the for f := range filenames loop never completes. However, just closing the input chan means that all goroutines launched in the loop would get stuck at the line sizes &lt;- info.Size() until the loop ends. Not a problem in this case but if the input can be huge it could be (then you'd probably want to limit the number of concurrent workers too). So it makes sense to have the main loop in a goroutine too so that the for size := range sizes loop can start consuming. Following should work:

func makeThumbnails5(filenames &lt;-chan string) int64 {
    sizes := make(chan int64)
    var wg sync.WaitGroup
    wg.Add(1)
    go func() {
        defer wg.Done()
        for f := range filenames {
            wg.Add(1)
            // worker
            go func(f string) {
                defer wg.Done()
                thumb, err := thumbnail.ImageFile(f)
                if err != nil {
                    log.Println(err)
                    return
                }
                info, _ := os.Stat(thumb)
                sizes &lt;- info.Size()
            }(f)
        }
    }()

    // closer
    go func() {
        wg.Wait()
        close(sizes)
    }()

    var total int64
    for size := range sizes {
        total += size
    }

    return total
}

The implementation of the main has a similar problem that if the input is huge you're essentially load it all into memory (buffered chan) before passing it on to be processed. Perhaps something like following is better

func main() {
    ch := make(chan string)
    go func(thumbnails []string) {
        defer close(ch)
        for _, val := range thumbnails {
            ch &lt;- val
        }
    }(os.Args[1:])
    makeThumbnails5(ch)
}

huangapple
  • 本文由 发表于 2022年12月28日 06:23:17
  • 转载请务必保留本文链接:https://go.coder-hub.com/74934599.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定