最佳实践是如何同步通道和等待组?

huangapple go评论74阅读模式
英文:

What's the best practice to synchronise channels and wait groups?

问题

同步等待组(wait group)和通道(channel)的最佳实践是什么?我想在一个循环中处理消息并进行阻塞,但是将通道的关闭委托给另一个 Go 协程似乎是一个奇怪的解决方案。

func Crawl(url string, depth int, fetcher Fetcher) {
    ch := make(chan string)

    var waitGroup sync.WaitGroup
    waitGroup.Add(1)
    go crawlTask(&waitGroup, ch, url, depth, fetcher)

    go func() {
        waitGroup.Wait()
        close(ch)
    }()

    for message := range ch {
        // 在这里处理消息
        fmt.Println(message)
    }
}

func crawlTask(waitGroup *sync.WaitGroup, ch chan string, url string, depth int, fetcher Fetcher) {
    defer waitGroup.Done()

    if depth <= 0 {
        return
    }
    body, urls, err := fetcher.Fetch(url)

    if err != nil {
        ch <- err.Error()
        return
    }
    ch <- fmt.Sprintf("found: %s %q\n", url, body)
    for _, u := range urls {
        waitGroup.Add(1)
        go crawlTask(waitGroup, ch, u, depth-1, fetcher)
    }
}

func main() {
    Crawl("http://golang.org/", 4, fetcher)
}

以上代码是从 https://tour.golang.org/concurrency/10 网页爬虫示例中截取的。

英文:

What's the best practice to synchronise wait groups and channels? I want to handle messages and block on a loop, and it appears that delegating the closing of the channel to another go routine seems to be a weird solution?

func Crawl(url string, depth int, fetcher Fetcher) {
	ch := make(chan string)

	var waitGroup sync.WaitGroup
	waitGroup.Add(1)
	go crawlTask(&amp;waitGroup, ch, url, depth, fetcher)

	go func() {
		waitGroup.Wait()
		close(ch)
	}()

	for message := range ch {
		// I want to handle the messages here
		fmt.Println(message)
	}
}

func crawlTask(waitGroup *sync.WaitGroup, ch chan string, url string, depth int, fetcher Fetcher) {
	defer waitGroup.Done()

	if depth &lt;= 0 {
		return
	}
	body, urls, err := fetcher.Fetch(url)

	if err != nil {
		ch &lt;- err.Error()
		return
	}
	ch &lt;- fmt.Sprintf(&quot;found: %s %q\n&quot;, url, body)
	for _, u := range urls {
     	waitGroup.Add(1)
		go crawlTask(waitGroup, ch, u, depth-1, fetcher)
	}
}
func main() {
	Crawl(&quot;http://golang.org/&quot;, 4, fetcher)
}

// truncated from https://tour.golang.org/concurrency/10 webCrawler

答案1

得分: 4

作为使用 waitgroup 和额外 goroutine 的替代方案,你可以使用一个单独的通道来结束 goroutine

这在 Go 中也是惯用的做法。它涉及使用 select 控制组进行阻塞

因此,你需要使用 make 创建一个新的通道,通常使用空结构体作为其值(例如 closeChan := make(chan struct{})),当关闭通道(close(closeChan))时,将结束 goroutine 本身。

你可以使用 select 来阻塞,直到接收到数据或通道关闭。

Crawl 函数中的代码可能如下所示:

for { // 替代遍历一个将要关闭的通道
    select {
    case message := <-ch:
        // 处理消息
    case <-closeChan:
        break // 退出 goroutine,也可以使用 return
    }
}

然后在 crawlTask 函数中,当你返回时(我猜这是你想要结束另一个 goroutine 并停止处理消息的时候),你可以关闭 closeChan(作为另一个参数传入,就像 ch 一样)。

if depth <= 0 {
    close(closeChan)
    return
}
英文:

As an alternative to using a waitgroup and extra goroutine, you can use a separate channel for ending goroutines.

This is (also) idiomatic in Go. It involves blocking using a select control group.

So you'd have to make a new channel, typically with an empty struct as it's value (eg closeChan := make(chan struct{}) which, when closed (close(closeChan)) would end the goroutine itself.

Instead of ranging over a chan, you can use a select to block until either fed data or closed.

The code in Crawl could look something like this:

for { // instead of ranging over a to-be closed chan
    select {
    case message := &lt;-ch:
        // handle message
    case &lt;-closeChan:
        break // exit goroutine, can use return instead
    }
}

And then in crawlTask, you could close the closeChan (passed in as another parameter, like ch when you return (I figure that's when you want the other goroutine to end, and stop handling messages?)

if depth &lt;= 0 {
    close(closeChan)
    return
}

答案2

得分: 2

使用单独的'closer' go-routine可以避免死锁。

如果等待/关闭操作在主go-routine中的for-range循环之前,它将永远不会结束,因为所有的'worker' go-routine都会在通道上没有接收者的情况下阻塞。如果将其放置在主go-routine中的for-range循环之后,它将无法访问,因为循环将在没有人关闭通道的情况下阻塞。

这个解释来自于《Go编程语言》书籍(8.5 并行循环)的借鉴。

英文:

Using a separate 'closer' go-routine prevents a deadlock.

If the wait/close operation were in the main go-routine before the for-range loop, it would never end, because all of the 'worker' go-routines would block in the absence of a receiver on the channel. And if it were placed in the main go-routine after the for-range loop, it would be unreachable, because the loop would block with no-one to close the channel.

This explanation was borrowed from 'The Go Programming Language' book (8.5 Looping in parallel).

huangapple
  • 本文由 发表于 2016年12月18日 00:09:04
  • 转载请务必保留本文链接:https://go.coder-hub.com/41200505.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定