英文:
How to check for goroutine completion while actively reading from channel?
问题
在一个句子中很难表达这个问题。以下是情况:
我正在尝试生成一组goroutine来递归遍历目录并查找匹配的文件。然后我收集这些文件并继续处理它们。然而,问题在于我不知道每个例程会找到多少文件,所以我很难弄清楚如何让主线程在所有例程完成后退出。
我可以让通道缓冲区非常大,但这不是一个好的解决方案,这个工具不需要100%的健壮性,但足够好,不会一直出错。而且有可能会找到很多文件。
// 启动一个例程来遍历每个目录
fpchan := make(chan string, 100)
for _, dir := range numDirs {
fmt.Printf("在%s中搜索文件\n", dir)
go findLogs(searchString, dir, fpchan)
}
// 从通道中收集文件路径
files := make([]string, 0, maxLogs)
for file := range fpchan { // 当一切都完成时,我会被卡住,没有东西可以接收
if len(files) <= cap(files) {
files.append(file)
} else {
fmt.Println("达到最大日志文件数%d\n", maxLogs)
}
}
等待组(waitgroup)并不适用,因为通道可能会填满,而例程会被阻塞(因为我事先不知道有多少结果)。
将一个空字符串作为信号发送到通道中,作为goroutine表示“完成”的方式是否可行?如下所示:
// 从通道中收集文件路径
files := make([]string, 0, maxLogs)
for file := range fpchan {
if file == "" {
fmt.Println("例程已完成")
numRunning--
if numRunning == 0 {
break
}
continue
}
if len(files) <= cap(files) {
files.append(file)
} else {
fmt.Println("达到最大日志文件数%d\n", maxLogs)
}
}
对于这种情况,由于空文件路径是无效的,它可以作为一种简单的信号。但这种方法感觉像是一个权宜之计,应该有更好的解决方案。
有没有一种不是非常复杂(使用大量额外的通道、非阻塞接收等)的方法来解决这个问题?如果比这个复杂得多,我会选择按顺序执行它们,但我认为这是一个很好的利用并发的机会。
英文:
Had a hard time expressing this question in a sentence. Here is the situation:
I'm trying to spawn off a set of goroutines to recurse over directories and find matching files. Then I collect those files and continue on to process them. However, the catch is that I don't know how many files each routine will find, so I'm having a hard time figuring out how to get the main thread to exit once all the routines are done
I could just make the channel buffer crazy big but that's not a good solution, this tool doesn't need to be 100% robust but good enough where it's not breaking all the time. Plus there's a chance it could turn up a lot of files
// start a routine to traverse each directory
fpchan := make(chan string, 100)
for _, dir := range numDirs {
fmt.Printf("Searching for file in %s\n", dir)
go findLogs(searchString, dir, fpchan)
}
// collect filepaths from channel
files := make([]string, 0, maxLogs)
for file := range fpchan { // I'LL GET STUCK WHEN EVERYTHING COMPLETES, NOTHING TO RECEIVE
if len(files) <= cap(files) {
files.append(file)
} else {
fmt.Println("Reached max logfile count of %d\n", maxLogs)
}
}
A waitgroup doesn't really work because the channel could fill up and the routines would be stuck (since I don't know how many results there will be ahead of time)
Is it kosher to send an empty string on the channel as a way for the goroutine to signal "complete"? Like the following:
// collect filepaths from channel
files := make([]string, 0, maxLogs)
for file := range fpchan {
if file == "" {
fmt.Println("goroutine finished")
numRunning--
if numRunning == 0 {
break
}
continue
}
if len(files) <= cap(files) {
files.append(file)
} else {
fmt.Println("Reached max logfile count of %d\n", maxLogs)
}
}
For this situation since an empty filepath would be invalid, it would work as a poor man's signal. It just feels like a hack for which there should be a better solution I would think
Any way to do this in a way that isn't horribly complicated (bunch of extra channels, non-blocking receives, etc)? If it's much more complicated than this I'd just do them in sequence but I thought it'd be a good chance to take advantage of concurrency
答案1
得分: 3
你可以在这里使用WaitGroup
。但是你需要在一个goroutine中等待waitgroup,在该goroutine中,当waitgroup完成时关闭通道,以便你的主循环终止:
var wg sync.WaitGroup
// 启动一个goroutine来遍历每个目录
fpchan := make(chan string, 100)
for _, dir := range numDirs {
fmt.Printf("在%s中搜索文件\n", dir)
wg.Add(1)
go func(dir string) {
findLogs(searchString, dir, fpchan)
wg.Done()
}(dir)
}
go func() {
wg.Wait()
close(fpchan)
}()
(你对`fpchan`的收集循环保持不变)
英文:
You can use a WaitGroup
here. But you need to wait for the waitgroup in a goroutine, and in that goroutine you close the channel when the waitgroup has completed, so that your main loop terminates:
var wg sync.WaitGroup
// start a routine to traverse each directory
fpchan := make(chan string, 100)
for _, dir := range numDirs {
fmt.Printf("Searching for file in %s\n", dir)
wg.Add(1)
go func(dir string) {
findLogs(searchString, dir, fpchan)
wg.Done()
}(dir)
}
go func() {
wg.Wait()
close(fpchan)
}()
(your collection loop over fpchan
remains the same)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论