一次唤醒0-N个休眠的goroutine的最佳方法

huangapple go评论81阅读模式
英文:

Best way to wake 0-N sleeping goroutines at once

问题

我正在编写一个程序,其中我启动N个工作线程(N是一个命令行参数),在任何时候0到N-1个线程可以等待另一个线程更新一个变量。线程等待这个事件的最佳方式是什么?一个线程通知其他所有线程该事件发生的最佳方式是什么?每个线程都会多次发送这个事件。

sync.Cond 不适用,因为线程在从睡眠中唤醒时不需要锁定资源。sync.WaitGroup 不适用,因为我不知道要调用多少次 wg.Done()

解决方案1:我可以使用 sync.Mutex,让最终会通知其他线程的线程获取锁并释放锁来通知其他线程,但是当它们只需要从睡眠中唤醒、读取一个变量以查看该特定工作线程是否成为主线程,然后要么继续睡眠要么开始工作时,其他线程竞争锁似乎非常低效。

解决方案2:创建一个 sync.WaitGroup 的包装器,允许跟踪等待线程的数量,这样我就可以调用 wg.Add(-numWaitingThreads) 来唤醒它们。但是,编写此代码而不产生各种竞争条件似乎很麻烦。

解决方案3:在没有更好的想法出现之前,我将使用一个包含N个通道的列表,并且通知者会向除自己以外的所有通道发送非阻塞的消息。这真的是最佳方式吗?

更多细节:我给每个工作线程分配一些唯一的信用,并有一个中央变量表示“下一个要写入输出文件的信用”。当一个工作线程完成其对应信用ID的工作后,它需要执行以下操作:

for centralNextCreditID != creditID {
  wait_for_centralNextCreditID_to_change()
}
saveWorkToFile()
centralNextCreditID++
wake_other_threads_waiting_for_centralNextCreditID_to_change()
英文:

I'm writing a program where I start N (N is a command-line argument) worker threads, and at any time 0 to N-1 of them can be waiting on another to update a variable. What's the best way for the threads to wait for this event, and the best way for one of the threads to notify all the others at once of the event occurring? This event will be sent multiple times by each thread.
sync.Cond isn't appropriate because the threads don't need to lock a resource upon waking from sleep. sync.WaitGroup won't work because I don't know how many times to call wg.Done().

Solution #1: I could use a sync.Mutex and have the thread that will eventually notify the others acquire the lock and then unlock it to notify the others, but it seems really inefficient for the others to all fight over a lock when they all just need to pop out of sleep, read a variable to see if that particular worker is now the master, and then either go back to sleep or start working.

Solution #2: Create a wrapper for sync.WaitGroup that allows keeping track of the number of waiting threads so that I can call wg.Add(-numWaitingThreads) to wake them. This sounds like a headache to figure out how to code it without all sorts of race conditions.

Solution #3: Until someone comes up with a better idea, I'll be using a list of N channels and have the notifier non-blocking-send to all of the channels except its own. Is this really the best way?

More details: I give each worker some unique credits and have a central variable for "which credit is the next to be written to the output file". When a worker finishes its work for whichever credit ID it was working on, it needs to do the following:

for centralNextCreditID != creditID {
  wait_for_centralNextCreditID_to_change()
}
saveWorkToFile()
centralNextCreditID++
wake_other_threads_waiting_for_centralNextCreditID_to_change()

答案1

得分: 2

对我来说,似乎这是一个适合使用sync.Cond的案例。你可以使用*RWMutex.RLocker()作为Cond.L,这样一旦发送了Cond.Broadcast(),所有的goroutine都可以同时获取读锁。

此外,为了避免竞争条件,当更改"who's master"变量时,最好确保持有写锁,这样sync.Cond就更适合了。

英文:

To me it does seem like this is an appropriate use case for sync.Cond. You can use a *RWMutex.RLocker() for Cond.L so all goroutines can acquire the read lock simultaneously once the Cond.Broadcast() is sent.

Additionally, it may be worth making sure you hold a write lock when changing this "who's master" variable to avoid race conditions, which would make sync.Cond an even better fit.

答案2

得分: 1

wg在这种情况下可以使用。创建一个计数为1的wg,并将其传递给N个goroutine。让它们执行wg.Wait(),除了更新变量的那个goroutine。更新变量的goroutine在成功更新后调用wg.Done(),从而导致N个goroutine退出等待状态并开始执行后续操作。

英文:

>sync.WaitGroup won't work because I don't know how many times to call wg.Done().

wg can be used in this case. Make a wg with count 1 and pass this to the N goroutines. Make them wg.Wait(), except the one that updates the variable.
The goroutine updating the variable calls wg.Done() after successful update thus resulting in N goroutines to come out of wait and start executing further.

答案3

得分: 0

标题说你想唤醒0-N个休眠的goroutine,但问题的正文表明你只需要唤醒下一个id的goroutine(如果有等待的goroutine)。

以下是如何实现问题正文中描述的问题:

// waiter按照递增的id工作序列。
type waiter struct {
    mu      sync.Mutex
    id      int
    waiting map[int]chan struct{}
}

func NewWaiter(firstID int) *waiter {
    return &waiter{id: firstID, waiting: make(map[int]chan struct{})}
}

// wait等待id在序列中的轮次。
func (w *waiter) wait(id int) {
    w.mu.Lock()
    if w.id == id {
        // 这个id是下一个。无需操作。
        w.mu.Unlock()
        return
    }
    // 等待我们的轮次。
    c := make(chan struct{})
    w.waiting[id] = c
    w.mu.Unlock()
    <-c
}

// done表示上一个id的工作已完成。
func (w *waiter) done() {
    w.mu.Lock()
    w.id++
    c, ok := w.waiting[w.id]
    if ok {
        delete(w.waiting, w.id)
    }
    w.mu.Unlock()
    if ok {
        // 关闭通道使c接收到零值
        close(c)
    }
}

以下是如何使用它:

for _, creditID := range creditIDs {
    doWorkFor(creditID)
    waiter.wait(creditID)
    saveWorkToFile()
    waiter.done()
}
英文:

The title says that you want to wake 0-N sleeping goroutines, but the body of the question indicates that you only need to wake the goroutine for the next id (if there is a goroutine waiting).

Here's how to implement the problem described in the body of the question:

// waiter sequences work according to an incrementing id.
type waiter struct {
	mu      sync.Mutex
	id      int
	waiting map[int]chan struct{}
}

func NewWaiter(firstID int) *waiter {
	return &amp;waiter{id: firstID, waiting: make(map[int]chan struct{})}
}

// wait waits for id&#39;s turn in the sequence.
func (w *waiter) wait(id int) {
	w.mu.Lock()
	if w.id == id {
		// This id is next. Nothing to do.
		w.mu.Unlock()
		return
	}
	// Wait for our turn.
	c := make(chan struct{})
	w.waiting[id] = c
	w.mu.Unlock()
	&lt;-c
}

// done signals that the work for the previous id is done.
func (w *waiter) done() {
    w.mu.Lock()
	w.id++
	c, ok := w.waiting[w.id]
	if ok {
		delete(w.waiting, w.id)
	}
	w.mu.Unlock()
	if ok {
		// close cause c to receive a zero value
		close(c)
	}
}

Here's how to use it:

for _, creditID := range creditIDs {
    doWorkFor(creditID)
    waiter.wait(creditID)
    saveWorkToFile()
    waiter.done()
}

答案4

得分: -1

WaitGroup 是最好的选择。原因是它保持了它的“已发信号”状态,如果主线程过早发出信号,你将不会遇到死锁的问题。

如果你使用 Cond,存在这样的风险:主线程在工作线程调用 cond.Wait() 之前调用了 cond.Broadcast。由于 Cond 不会记住它是否已经发出信号,工作线程将一直等待事件发生。
这里有一个例子:https://go.dev/play/p/YLfvEGO2A18
主线程过早广播,工作线程陷入了死锁。

con.WaitGroup 的情况也是一样的:https://go.dev/play/p/R6_-ULo2eJ2
主线程过早释放了等待组,但没有发生死锁。

英文:

WaitGroup is the best option. The reason is that is keeps its signalled state and you are safe from deadlock if the main thread signals too early.

If you use Cond there is a risk that the main thread calls cond.Broadcast BEFORE the worker thread calls cond.Wait(). Since Cond doesn't remember that it was signalled, the worker thread will wait for the event to happen.
Here is an example: https://go.dev/play/p/YLfvEGO2A18
The main thread broadcasts too early, the worker threads run into a deadlock.

Same case with con.WaitGroup: https://go.dev/play/p/R6_-ULo2eJ2
The main thread releases the wait group too early, but there is no deadlock.

huangapple
  • 本文由 发表于 2022年8月24日 07:31:13
  • 转载请务必保留本文链接:https://go.coder-hub.com/73465941.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定