英文:
Understanding golang channels. All goroutines are asleep - deadlock [Tour of Go, Crawler]
问题
可能这就是一个从未处理过多线程程序的PHP开发人员开始学习golang和通道时的情况。
我正在进行Go之旅的最后一个练习,【练习:Web爬虫】(在此之前的练习都没有问题)
尽管我试图编写尽可能简单的代码,但我的Crawl方法看起来像这样:
func Crawl(url string, depth int, fetcher Fetcher) {
// 通过将初始URL传递给作业队列来启动爬取
Queue <- Job{
url,
depth,
}
// 确保我们关闭队列通道
defer close(Queue)
// 从队列中读取
for job := range Queue {
// 如果已获取或已达到深度底部,
// 立即继续选择下一个作业
if fetched.Has(job.Url) || job.Depth <= 0 {
continue
}
fres := fetcher.Fetch(job.Url)
fetched.Add(job.Url, fres)
for i := range fres.Urls {
// 将从当前URL获取的新URL发送到队列中
Queue <- Job{
fres.Urls[i], job.Depth - 1,
}
}
}
for _, res := range fetched.m {
fmt.Println(res)
}
}
go run
告诉我不应该编写任何 go
代码,而应返回到 PHP
:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.Crawl(0xf37c1, 0x12, 0x4, 0x1600e0, 0x104401c0, 0x104000f0)
/tmp/sandbox452918312/main.go:64 +0x80
main.main()
/tmp/sandbox452918312/main.go:87 +0x60
当然,我已经搜索了这个问题,结论通常是:“关闭你的通道”,我也这样做了(我是吗?)。
那么,有人能指出我在这里漏掉了什么吗?
完整的代码在这里:https://play.golang.org/p/-98SdVndD6
对于这个练习,最符合Go语言习惯的方式是什么?我找到了一些解决方案。
等等,哪一个对你来说是一个干净的解决方案?
另外,我应该仅仅使用通道和goroutines吗?
英文:
Probably this is what happens when a PHP-developer who has never dealt with multi-threaded programs starts to learn golang and channels.
I'm on the last exercise of Tour of Go, [Exercise: Web Crawler] (I had no problems with other exercises before this one)
I'm trying to write as simple code as possible though,
my Crawl method looks like this:
func Crawl(url string, depth int, fetcher Fetcher) {
// kick off crawling by passing initial Url to a Job queue
Queue <- Job{
url,
depth,
}
// make sure we close the Queue channel
defer close(Queue)
// read from the Queue
for job := range Queue {
// if fetched or has hit the bottom of depth,
// just continue right away to pick up next Job
if fetched.Has(job.Url) || job.Depth <= 0 {
continue
}
fres := fetcher.Fetch(job.Url)
fetched.Add(job.Url, fres)
for i := range fres.Urls {
// send new urls just fetched from current url in Job
// to the Queue
Queue <- Job{
fres.Urls[i], job.Depth - 1,
}
}
}
for _, res := range fetched.m {
fmt.Println(res)
}
}
go run
says I should not write any go
code and get back to PHP
:
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan send]:
main.Crawl(0xf37c1, 0x12, 0x4, 0x1600e0, 0x104401c0, 0x104000f0)
/tmp/sandbox452918312/main.go:64 +0x80
main.main()
/tmp/sandbox452918312/main.go:87 +0x60
Of course, I have googled this problem and the conclusion is usually: "close your chans", which I did (did I?).
So, can somebody point out what am I missing here?
The complete code is here: https://play.golang.org/p/-98SdVndD6
What is the most idiomatic golang way for this exercise? I've found a handful of them.
etc. which one seems to be a clean solution to you?
Also, should I use channels only with goroutines?
答案1
得分: 1
你正在“延迟”关闭队列。这意味着“当这个函数(Crawl)退出时关闭队列!”
然后你进入一个循环,该循环将阻塞直到以下两种情况之一发生:
- 接收到一个项目,或者
- “队列”被关闭
在开始时,队列中添加了一个“作业”(这将使循环运行一次),然后在第一次运行结束时,循环将再次阻塞,直到再次满足上述两个条件之一。
注意:第一次循环可能会向队列中添加更多的项目(从而导致更多的迭代),但在某个时候,循环的队列将耗尽,并且循环将再次阻塞,等待上述两个条件之一
然而,队列中永远不会再添加任何项目(因此#1失败),并且“队列”只有在此函数退出之后才关闭,而此函数直到循环退出之后才能退出(因此#2失败)。
**简而言之:**你的循环正在等待函数退出,而你的函数正在等待循环退出-死锁
英文:
You're 'defer'ring the closing of the queue. This means "Close the Queue when this function (Crawl) exits!"
Then you enter a loop that will block until it either:
- Receives an item OR
- 'Queue' is closed
There is a 'Job' added to the queue at the beginning (this would allow the loop to run a single time), then at the end of the first run, the loop will block until either of the above two conditions are met once again.
Note: The run through the first loop could potentially add more items to the queue (and therefore cause more iterations), at some point though, the loop's queue will be exhausted and the loop will once again block waiting for one of the above two conditions
However, there are never any more items added to the queue (so #1 fails) and the 'Queue' is only closed after this function exits, which cannot happen until after the loop exits (so #2 fails).
TLDR: Your loop is waiting for your function to exit, and your function is waiting for your loop to exit - Deadlock
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论