如何在主动从通道读取数据时检查 goroutine 的完成状态?

huangapple go评论80阅读模式
英文:

How to check for goroutine completion while actively reading from channel?

问题

在一个句子中很难表达这个问题。以下是情况:

我正在尝试生成一组goroutine来递归遍历目录并查找匹配的文件。然后我收集这些文件并继续处理它们。然而,问题在于我不知道每个例程会找到多少文件,所以我很难弄清楚如何让主线程在所有例程完成后退出。

我可以让通道缓冲区非常大,但这不是一个好的解决方案,这个工具不需要100%的健壮性,但足够好,不会一直出错。而且有可能会找到很多文件。

// 启动一个例程来遍历每个目录
fpchan := make(chan string, 100)
for _, dir := range numDirs {
    fmt.Printf("在%s中搜索文件\n", dir)
    go findLogs(searchString, dir, fpchan)
}

// 从通道中收集文件路径
files := make([]string, 0, maxLogs)
for file := range fpchan { // 当一切都完成时,我会被卡住,没有东西可以接收
    if len(files) <= cap(files) {
        files.append(file)
    } else {
        fmt.Println("达到最大日志文件数%d\n", maxLogs)
    }
}

等待组(waitgroup)并不适用,因为通道可能会填满,而例程会被阻塞(因为我事先不知道有多少结果)。

将一个空字符串作为信号发送到通道中,作为goroutine表示“完成”的方式是否可行?如下所示:

// 从通道中收集文件路径
files := make([]string, 0, maxLogs)
for file := range fpchan {
    if file == "" {
        fmt.Println("例程已完成")
        numRunning--
        if numRunning == 0 {
             break
        }
        continue
    }
    if len(files) <= cap(files) {
        files.append(file)
    } else {
        fmt.Println("达到最大日志文件数%d\n", maxLogs)
    }
}

对于这种情况,由于空文件路径是无效的,它可以作为一种简单的信号。但这种方法感觉像是一个权宜之计,应该有更好的解决方案。

有没有一种不是非常复杂(使用大量额外的通道、非阻塞接收等)的方法来解决这个问题?如果比这个复杂得多,我会选择按顺序执行它们,但我认为这是一个很好的利用并发的机会。

英文:

Had a hard time expressing this question in a sentence. Here is the situation:

I'm trying to spawn off a set of goroutines to recurse over directories and find matching files. Then I collect those files and continue on to process them. However, the catch is that I don't know how many files each routine will find, so I'm having a hard time figuring out how to get the main thread to exit once all the routines are done

I could just make the channel buffer crazy big but that's not a good solution, this tool doesn't need to be 100% robust but good enough where it's not breaking all the time. Plus there's a chance it could turn up a lot of files

// start a routine to traverse each directory
fpchan := make(chan string, 100)
for _, dir := range numDirs {
    fmt.Printf(&quot;Searching for file in %s\n&quot;, dir)
    go findLogs(searchString, dir, fpchan)
}

// collect filepaths from channel
files := make([]string, 0, maxLogs)
for file := range fpchan { // I&#39;LL GET STUCK WHEN EVERYTHING COMPLETES, NOTHING TO RECEIVE
    if len(files) &lt;= cap(files) {
        files.append(file)
    } else {
        fmt.Println(&quot;Reached max logfile count of %d\n&quot;, maxLogs)
    }
}

A waitgroup doesn't really work because the channel could fill up and the routines would be stuck (since I don't know how many results there will be ahead of time)

Is it kosher to send an empty string on the channel as a way for the goroutine to signal "complete"? Like the following:

// collect filepaths from channel
files := make([]string, 0, maxLogs)
for file := range fpchan {
    if file == &quot;&quot; {
        fmt.Println(&quot;goroutine finished&quot;)
        numRunning--
        if numRunning == 0 {
             break
        }
        continue
    }
    if len(files) &lt;= cap(files) {
        files.append(file)
    } else {
        fmt.Println(&quot;Reached max logfile count of %d\n&quot;, maxLogs)
    }
}

For this situation since an empty filepath would be invalid, it would work as a poor man's signal. It just feels like a hack for which there should be a better solution I would think

Any way to do this in a way that isn't horribly complicated (bunch of extra channels, non-blocking receives, etc)? If it's much more complicated than this I'd just do them in sequence but I thought it'd be a good chance to take advantage of concurrency

答案1

得分: 3

你可以在这里使用WaitGroup。但是你需要在一个goroutine中等待waitgroup,在该goroutine中,当waitgroup完成时关闭通道,以便你的主循环终止:

var wg sync.WaitGroup
// 启动一个goroutine来遍历每个目录
fpchan := make(chan string, 100)
for _, dir := range numDirs {
    fmt.Printf("在%s中搜索文件\n", dir)
    wg.Add(1)
    go func(dir string) {
        findLogs(searchString, dir, fpchan)
        wg.Done()
    }(dir)
}

go func() {
    wg.Wait()
    close(fpchan)
}()

你对`fpchan`的收集循环保持不变
英文:

You can use a WaitGroup here. But you need to wait for the waitgroup in a goroutine, and in that goroutine you close the channel when the waitgroup has completed, so that your main loop terminates:

var wg sync.WaitGroup
// start a routine to traverse each directory
fpchan := make(chan string, 100)
for _, dir := range numDirs {
	fmt.Printf(&quot;Searching for file in %s\n&quot;, dir)
	wg.Add(1)
	go func(dir string) {
		findLogs(searchString, dir, fpchan)
		wg.Done()
	}(dir)
}

go func() {
	wg.Wait()
	close(fpchan)
}()

(your collection loop over fpchan remains the same)

huangapple
  • 本文由 发表于 2022年9月10日 16:10:47
  • 转载请务必保留本文链接:https://go.coder-hub.com/73670269.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定