理解golang通道。所有的goroutine都处于休眠状态-死锁[Tour of Go, Crawler]

huangapple go评论79阅读模式
英文:

Understanding golang channels. All goroutines are asleep - deadlock [Tour of Go, Crawler]

问题

可能这就是一个从未处理过多线程程序的PHP开发人员开始学习golang和通道时的情况。

我正在进行Go之旅的最后一个练习,【练习:Web爬虫】(在此之前的练习都没有问题)

尽管我试图编写尽可能简单的代码,但我的Crawl方法看起来像这样:

func Crawl(url string, depth int, fetcher Fetcher) {
    // 通过将初始URL传递给作业队列来启动爬取
    Queue <- Job{
        url,
        depth,
    }

    // 确保我们关闭队列通道
    defer close(Queue)

    // 从队列中读取
    for job := range Queue {
        // 如果已获取或已达到深度底部,
        // 立即继续选择下一个作业
        if fetched.Has(job.Url) || job.Depth <= 0 {
            continue
        }
        fres := fetcher.Fetch(job.Url)
        fetched.Add(job.Url, fres)
        for i := range fres.Urls {
            // 将从当前URL获取的新URL发送到队列中
            Queue <- Job{
                fres.Urls[i], job.Depth - 1,
            }
        }
    }

    for _, res := range fetched.m {
        fmt.Println(res)
    }
}

go run 告诉我不应该编写任何 go 代码,而应返回到 PHP

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.Crawl(0xf37c1, 0x12, 0x4, 0x1600e0, 0x104401c0, 0x104000f0)
    /tmp/sandbox452918312/main.go:64 +0x80
main.main()
    /tmp/sandbox452918312/main.go:87 +0x60

当然,我已经搜索了这个问题,结论通常是:“关闭你的通道”,我也这样做了(我是吗?)。

那么,有人能指出我在这里漏掉了什么吗?

完整的代码在这里:https://play.golang.org/p/-98SdVndD6

对于这个练习,最符合Go语言习惯的方式是什么?我找到了一些解决方案。

等等,哪一个对你来说是一个干净的解决方案?

另外,我应该仅仅使用通道和goroutines吗?

英文:

Probably this is what happens when a PHP-developer who has never dealt with multi-threaded programs starts to learn golang and channels.

I'm on the last exercise of Tour of Go, [Exercise: Web Crawler] (I had no problems with other exercises before this one)

I'm trying to write as simple code as possible though,
my Crawl method looks like this:

func Crawl(url string, depth int, fetcher Fetcher) {
    // kick off crawling by passing initial Url to a Job queue
	Queue &lt;- Job{
		url,
		depth,
	}

	// make sure we close the Queue channel
    defer close(Queue)

    // read from the Queue
	for job := range Queue {
        // if fetched or has hit the bottom of depth,
        // just continue right away to pick up next Job
		if fetched.Has(job.Url) || job.Depth &lt;= 0 {
			continue
		}
		fres := fetcher.Fetch(job.Url)
		fetched.Add(job.Url, fres)
		for i := range fres.Urls {
            // send new urls just fetched from current url in Job
            // to the Queue
			Queue &lt;- Job{
				fres.Urls[i], job.Depth - 1,
			}
		}
	}

	for _, res := range fetched.m {
		fmt.Println(res)
	}
}

go run says I should not write any go code and get back to PHP:

fatal error: all goroutines are asleep - deadlock!

goroutine 1 [chan send]:
main.Crawl(0xf37c1, 0x12, 0x4, 0x1600e0, 0x104401c0, 0x104000f0)
    /tmp/sandbox452918312/main.go:64 +0x80
main.main()
    /tmp/sandbox452918312/main.go:87 +0x60

Of course, I have googled this problem and the conclusion is usually: "close your chans", which I did (did I?).

So, can somebody point out what am I missing here?

The complete code is here: https://play.golang.org/p/-98SdVndD6

What is the most idiomatic golang way for this exercise? I've found a handful of them.

etc. which one seems to be a clean solution to you?

Also, should I use channels only with goroutines?

答案1

得分: 1

你正在“延迟”关闭队列。这意味着“当这个函数(Crawl)退出时关闭队列!”

然后你进入一个循环,该循环将阻塞直到以下两种情况之一发生:

  1. 接收到一个项目,或者
  2. “队列”被关闭

在开始时,队列中添加了一个“作业”(这将使循环运行一次),然后在第一次运行结束时,循环将再次阻塞,直到再次满足上述两个条件之一。

注意:第一次循环可能会向队列中添加更多的项目(从而导致更多的迭代),但在某个时候,循环的队列将耗尽,并且循环将再次阻塞,等待上述两个条件之一

然而,队列中永远不会再添加任何项目(因此#1失败),并且“队列”只有在此函数退出之后才关闭,而此函数直到循环退出之后才能退出(因此#2失败)。

**简而言之:**你的循环正在等待函数退出,而你的函数正在等待循环退出-死锁

英文:

You're 'defer'ring the closing of the queue. This means "Close the Queue when this function (Crawl) exits!"

Then you enter a loop that will block until it either:

  1. Receives an item OR
  2. 'Queue' is closed

There is a 'Job' added to the queue at the beginning (this would allow the loop to run a single time), then at the end of the first run, the loop will block until either of the above two conditions are met once again.

Note: The run through the first loop could potentially add more items to the queue (and therefore cause more iterations), at some point though, the loop's queue will be exhausted and the loop will once again block waiting for one of the above two conditions

However, there are never any more items added to the queue (so #1 fails) and the 'Queue' is only closed after this function exits, which cannot happen until after the loop exits (so #2 fails).

TLDR: Your loop is waiting for your function to exit, and your function is waiting for your loop to exit - Deadlock

huangapple
  • 本文由 发表于 2017年5月3日 08:57:46
  • 转载请务必保留本文链接:https://go.coder-hub.com/43749424.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定