英文:
go routine panics when i return multiple errors
问题
我正在玩弄工作池,并且我想在返回错误之前将所有来自工作池的错误合并。我已经编写了一个示例代码,但是我遇到了死锁的问题。
我想要实现什么?
一个客户端发送100个请求,我想首先将这些请求添加到作业队列中,并将其分派给n个后台执行任务的Go协程,如果有错误发生,我希望在将所有错误发送给客户端之前累积所有这些错误。我已经编写了一段代码,有人可以解释一下问题出在哪里,以及如何解决死锁问题吗?
package main
import (
"context"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/apex/log"
"github.com/hashicorp/go-multierror"
)
type Manager struct {
taskChan chan int
wg *sync.WaitGroup
QuitChan chan bool
ErrorChan chan error
busyWorkers int64
}
func main() {
fmt.Println("Hello, 世界")
m := New()
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
//defer cancel()
for i := 0; i < 3; i++ {
m.wg.Add(1)
go m.run(ctx, test)
}
for i := 1; i < 5; i++ {
m.taskChan <- i
}
close(m.taskChan)
go func(*Manager) {
if len(m.taskChan) == 0 {
m.QuitChan <- true
}
}(m)
var errors error
for {
select {
case err := <-m.ErrorChan:
errors = multierror.Append(errors, err)
if m.busyWorkers == int64(0) {
break
}
default:
fmt.Println("hello")
}
}
m.wg.Wait()
fmt.Println(errors)
}
func New() *Manager {
return &Manager{taskChan: make(chan int),
wg: new(sync.WaitGroup),
QuitChan: make(chan bool),
ErrorChan: make(chan error),
}
}
func (m *Manager) run(ctx context.Context, fn func(a, b int) error) {
defer m.wg.Done()
defer fmt.Println("finished working")
for {
select {
case t, ok := <-m.taskChan:
if ok {
atomic.AddInt64(&m.busyWorkers, 1)
err := fn(t, t)
if err != nil {
m.ErrorChan <- err
}
atomic.AddInt64(&m.busyWorkers, -1)
}
case <-ctx.Done():
log.Infof("closing channel %v", ctx.Err())
return
case <-m.QuitChan:
return
}
}
}
// this can return error or not, this is the main driver func, but i'm propagating
//errors so that i can understand where i am going wrong
func test(a, b int) error {
fmt.Println(a, b)
return fmt.Errorf("dummy error %v", a)
}
请注意,我只会返回翻译好的代码部分,不会回答关于翻译的问题。
英文:
I am playing around with worker pools and i want to consolidate all errors from worker pools before i return the error. I've written a sample code but i am entering a deadlock.
What am i trying to achieve?
a client send 100 requests,i want to first add those requests to a job queue and dispatch it to n number of go routines that does tasks in background , if at all there are errors i want to accumulate all these errors before i send all errors to the client. I have written a snippet, can someone explain what's wrong and how to mitigate the deadlock.
package main
import (
"context"
"fmt"
"sync"
"sync/atomic"
"time"
"github.com/apex/log"
"github.com/hashicorp/go-multierror"
)
type Manager struct {
taskChan chan int
wg *sync.WaitGroup
QuitChan chan bool
ErrorChan chan error
busyWorkers int64
}
func main() {
fmt.Println("Hello, 世界")
m := New()
ctx, _ := context.WithTimeout(context.Background(), 5*time.Second)
//defer cancel()
for i := 0; i < 3; i++ {
m.wg.Add(1)
go m.run(ctx, test)
}
for i := 1; i < 5; i++ {
m.taskChan <- i
}
close(m.taskChan)
go func(*Manager) {
if len(m.taskChan) == 0 {
m.QuitChan <- true
}
}(m)
var errors error
for {
select {
case err := <-m.ErrorChan:
errors = multierror.Append(errors, err)
if m.busyWorkers == int64(0) {
break
}
default:
fmt.Println("hello")
}
}
m.wg.Wait()
fmt.Println(errors)
}
func New() *Manager {
return &Manager{taskChan: make(chan int),
wg: new(sync.WaitGroup),
QuitChan: make(chan bool),
ErrorChan: make(chan error),
}
}
func (m *Manager) run(ctx context.Context, fn func(a, b int) error) {
defer m.wg.Done()
defer fmt.Println("finished working")
for {
select {
case t, ok := <-m.taskChan:
if ok {
atomic.AddInt64(&m.busyWorkers, 1)
err := fn(t, t)
if err != nil {
m.ErrorChan <- err
}
atomic.AddInt64(&m.busyWorkers, -1)
}
case <-ctx.Done():
log.Infof("closing channel %v", ctx.Err())
return
case <-m.QuitChan:
return
}
}
}
// this can return error or not, this is the main driver func, but i'm propagating
//errors so that i can understand where i am going wrong
func test(a, b int) error {
fmt.Println(a, b)
return fmt.Errorf("dummy error %v", a)
}
答案1
得分: 1
你有3个工人,他们都返回错误。
你的主线程试图将5个任务放入队列中。一旦前3个任务被工人接收,主线程就会被阻塞,等待一个新的工人在taskChan
上接收,而你的3个工人都被阻塞在试图在ErrorChan
上发送数据。
换句话说,发生了死锁。
也许你想把taskChan
变成一个带缓冲区的通道?这样你就可以在缓冲区满之前无阻塞地发送数据。
taskChan: make(chan int, 10)
英文:
You have 3 workers who all return errors.
Your main thread tries to put 5 jobs in the queue. Once the first 3 has been taken by your workers, the main thread is stuck waiting for a new worker to receive on taskChan
and all your 3 workers are stuck trying to send data on ErrorChan
.
In other words, deadlock.
Maybe you wanted to make taskChan
a buffered channel? That way you can send data on it until the buffer is full without blocking.
taskChan: make(chan int, 10)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论