英文:
Deadlock error in golang
问题
这个程序出现死锁的原因是在printHashes
函数中,你使用了for range
循环来接收files
通道中的值,但是没有在循环体内消费这些值。这导致了files
通道一直处于阻塞状态,没有其他的goroutine可以往通道中发送数据,最终导致了死锁。
要解决这个问题,你可以修改printHashes
函数,将for range
循环改为普通的for
循环,并在循环体内消费通道中的值。修改后的代码如下:
func printHashes(files <-chan string, wg *sync.WaitGroup) {
for file := range files {
fmt.Println(file)
}
wg.Done()
}
这样修改之后,程序就不会出现死锁错误了。
英文:
I recently looked at go and got hooked, it looks so interesting! After completing the tutorial I wanted to build something by myself: I want to list all of my songs from my music library. I think I can profit from go's concurrency here. While on routine is walking down the directory tree it pushes music files (path to those files) into a channel which are then picked up by another routine that reads the ID3 tags, so I don't have to wait until every file has been found.
This is my simple and naive approach:
package main
import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
)
const searchPath = "/Users/luma/Music/test" // 5GB of music.
func main() {
files := make(chan string)
var wg sync.WaitGroup
wg.Add(2)
go printHashes(files, &wg)
go searchFiles(searchPath, files, &wg)
wg.Wait()
}
func searchFiles(searchPath string, files chan<- string, wg *sync.WaitGroup) {
visit := func(path string, f os.FileInfo, err error) error {
if !f.IsDir() && strings.Contains(".mp4.mp3.flac", filepath.Ext(f.Name())) {
files <- path
}
return err
}
if err := filepath.Walk(searchPath, visit); err != nil {
fmt.Println(err)
}
wg.Done()
}
func printHashes(files <-chan string, wg *sync.WaitGroup) {
for range files {
fmt.Println(<-files)
}
wg.Done()
}
This program doesn't read the tags, yet. Instead it just prints the file path. This works, it lists all music files extremely fast! But I see this error after the program finishes:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [semacquire]:
sync.runtime_Semacquire(0xc42007205c)
/usr/local/Cellar/go/1.7.4_2/libexec/src/runtime/sema.go:47 +0x30
sync.(*WaitGroup).Wait(0xc420072050)
/usr/local/Cellar/go/1.7.4_2/libexec/src/sync/waitgroup.go:131 +0x97
main.main()
/Users/luma/Code/Go/src/github.com/LuMa/test/main.go:22 +0xfa
goroutine 17 [chan receive]:
main.printHashes(0xc42008e000, 0xc420072050)
/Users/luma/Code/Go/src/github.com/LuMa/test/main.go:42 +0xb4
created by main.main
/Users/luma/Code/Go/src/github.com/LuMa/test/main.go:19 +0xab
exit status 2
What is causing the deadlock?
答案1
得分: 2
因为你需要关闭files
通道。
在你的情况下,你没有关闭它,所以for range files { fmt.Println(<-files) }
会等待从files
通道获取值。所以printHashes
中的wg.Done()
永远不会完成。
func searchFiles(searchPath string, files chan<- string, wg *sync.WaitGroup) {
visit := func(path string, f os.FileInfo, err error) error {
if !f.IsDir() && strings.Contains(".mp4.mp3.flac", filepath.Ext(f.Name())) {
files <- path
}
return err
}
if err := filepath.Walk(searchPath, visit); err != nil {
fmt.Println(err)
}
wg.Done()
close(files) // 关闭通道,因为你不再往通道中放东西了。
}
英文:
Because you need close files
channel.
In your case, you don't close it, so
for range files {
will wait get value from
fmt.Println(<-files)
}files
channel. so wg.Done()
will never done in printHashes
.
func searchFiles(searchPath string, files chan<- string, wg *sync.WaitGroup) {
visit := func(path string, f os.FileInfo, err error) error {
if !f.IsDir() && strings.Contains(".mp4.mp3.flac", filepath.Ext(f.Name())) {
files <- path
}
return err
}
if err := filepath.Walk(searchPath, visit); err != nil {
fmt.Println(err)
}
wg.Done()
close(files) // close the chanel, because you don't put thing into the channel anymore.
}
答案2
得分: 1
在searchFiles
函数中,当发送完成后,你想要关闭files
通道。这个约定被称为发送者关闭(接收者不关闭)。另外,移除对wg.Done()
的调用,因为你还没有完成...通道中可能仍然有项目。
close(files)
会向for range files
发送关闭信号并退出循环,这将调用你的wg.Done()
来向主函数发送所有操作已完成的信号。
(在移动设备上未经测试)
package main
import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
)
const searchPath = "/Users/luma/Music/test" // 5GB的音乐。
func main() {
files := make(chan string)
var wg sync.WaitGroup
wg.Add(1)
go printHashes(files)
go searchFiles(searchPath, files, &wg)
wg.Wait()
}
func searchFiles(searchPath string, files chan<- string) {
visit := func(path string, f os.FileInfo, err error) error {
if !f.IsDir() && strings.Contains(".mp4.mp3.flac", filepath.Ext(f.Name())) {
files <- path
}
return err
}
if err := filepath.Walk(searchPath, visit); err != nil {
fmt.Println(err)
}
close(files)
}
func printHashes(files <-chan string, wg *sync.WaitGroup) {
defer wg.Done()
for range files {
fmt.Println(<-files)
}
}
请注意,虽然这可能看起来很快,但使用单个goroutine是可以的,并且不会阻塞主goroutine。但是,如果你尝试在多个goroutine中读取多个文件的id3标签,你可能不会获得任何优势-它们都将在系统调用级别共享相同的文件I/O锁。唯一有利的情况是,如果数据处理远远超过文件I/O锁定(例如,计算中有大量的处理,因为处理速度比系统调用锁定要快得多)。
欢迎加入Go社区!
英文:
Within searchFiles
, you want to close(files)
when done sending. This convention is called sender-closes (receivers never close). Also, remove the call to wg.Done()
as you are not done... There could still be items on the channel.
The close(files)
will signal the for range files
to close and exit the loop, which will call your wg.Done()
to signal the main function that everything is done.
(Untested on mobile)
package main
import (
"fmt"
"os"
"path/filepath"
"strings"
"sync"
)
const searchPath = "/Users/luma/Music/test" // 5GB of music.
func main() {
files := make(chan string)
var wg sync.WaitGroup
wg.Add(1)
go printHashes(files)
go searchFiles(searchPath, files, &wg)
wg.Wait()
}
func searchFiles(searchPath string, files chan<- string) {
visit := func(path string, f os.FileInfo, err error) error {
if !f.IsDir() && strings.Contains(".mp4.mp3.flac", filepath.Ext(f.Name())) {
files <- path
}
return err
}
if err := filepath.Walk(searchPath, visit); err != nil {
fmt.Println(err)
}
close(files)
}
func printHashes(files <-chan string, wg *sync.WaitGroup) {
defer wg.Done()
for range files {
fmt.Println(<-files)
}
}
Note that while this may seem fast, using a single goroutine is fine and unblocks the main goroutine too. But, you may not gain any advantage if you try to read multiple files for id3 tags in multiple goroutines - they will all share the same file i/o lock at the syscall level. The only way that would advantageous would be if the processing of data far out weighs the file i/o locking (e.g. something big in computation, because processing is far faster than syscall locks).
PS, welcome to the Go community!
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论