使用一个通道从多个goroutine接收结果的Go代码。

huangapple go评论80阅读模式
英文:

Go: using one channels to receive results from multiple goroutines

问题

我有一个包含数千条记录的CSV文件。
我在每个goroutine中处理每条记录,并希望将所有处理后的记录结果收集到一个切片中,以便将它们写入另一个文件。
代码的简化版本如下:

var wg sync.WaitGroup

func main() {
    c := make(chan string, 1)
    csvfile, err := os.Open("data.csv")
    reader := csv.NewReader(bufio.NewReader(csvfile))
    //...
    for {
        line, err := reader.Read()
        // ...
        wg.Add(1)
        go processRecord(line, c)
    }
    //...
    results := make([]string, 0)
    for r := range c { // ERROR HERE: fatal error: all goroutines are asleep - deadlock!
        results = append(results, r)
    }
    // write results to file
    wg.Wait()
    close(c)
}

func processRecord(data []string, c chan string) {
    defer wg.Done()
    // ...
    c <- *result
}

在for循环中,我获取了所有的结果,但是在获取到最后一个结果后,循环继续执行,然后出现了这个错误:

fatal error: all goroutines are asleep - deadlock! 

这段代码有什么问题?我在Stack Overflow上尝试了不同的问题和答案,但都没有成功解决。

英文:

I have a CSV file with thousands of records.
I process each record in a goroutine and I want to gather all the results of processed records in a slice to write them down in another file.
Simplified version of my code is:

var wg sync.WaitGroup

func main() {
    c := make(chan string, 1)
    csvfile, err := os.Open(&quot;data.csv&quot;)
    reader := csv.NewReader(bufio.NewReader(csvfile))
    //...
    for {
        line, err := reader.Read()
        // ...
        wg.Add(1)
        go processRecord(line, c)
    }
    //...
    results := make([]string, 0)
	for r := range c { // ERROR HERE: fatal error: all goroutines are asleep - deadlock!
		results = append(results , r)
	}
    // write results to file
	wg.Wait()
	close(c)
}

func processRecord(data []string, c chan string) {
	defer wg.Done()
    // ...
    c &lt;- *result
}

In for loop I get all the results but loop continues after getting last result and I get this error:

fatal error: all goroutines are asleep - deadlock! 

What is wrong with this code? I tried different QAs on SO without any success.

答案1

得分: 3

for循环只有在通道关闭后才会终止。你是在for循环终止后关闭通道,因此导致了死锁。

你可以通过将for循环放入一个goroutine中来修复:

results := make([]string, 0)
done := make(chan struct{})
go func() {
    for r := range c {
        results = append(results, r)
    }
    close(done)
}()
wg.Wait()
close(c)
<-done
// 读取结果
英文:

The for-loop will only terminate once the channel is closed. You are closing the channel after for-loop terminates, hence the deadlock.

You can fix by putting the for-loop into a goroutine:

results := make([]string, 0)
done:=make(chan struct{})
go func() {
    for r := range c {
        results = append(results , r)
    }
    close(done)
}()
wg.Wait()
close(c)
&lt;-done
// read results

答案2

得分: 0

你可以使用互斥锁(Mutex)来处理这个异常。以下是修复后的代码:

for r := range c {
    mutex.Lock()
    results = append(results, r)
    mutex.Unlock()
}

通过使用互斥锁,可以确保在对results进行操作时只有一个goroutine可以访问它,从而避免了死锁的问题。

英文:
for r := range c { // ERROR HERE: fatal error: all goroutines are asleep - deadlock!
    mutex.Lock()
    results = append(results , r)
    mutex.Unlock()
}

goroutines are asleep - deadlock! because its written at the same time. Use Mutex can be handle this exception

huangapple
  • 本文由 发表于 2022年1月27日 07:52:33
  • 转载请务必保留本文链接:https://go.coder-hub.com/70871684.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定