将连续测试分散到4个Go协程中,并在其中一个失败时终止所有测试。

huangapple go评论95阅读模式
英文:

Spread sequential tests into 4 go routines and terminate all if one fails

问题

假设我有一个简单的循环,像这样进行顺序测试。

for f := 1; f <= 1000; f++ {
    if doTest(f) {
        break
    }
}

我循环遍历一系列数字,并对每个数字进行测试。如果某个数字的测试失败,我会中断循环并退出主线程。很简单。

现在,我想要正确地将测试数字分批次地提供给四个或多个goroutine。基本上,我想要将数字从1到1000分成4个一组(或任意数量的goroutine)进行测试。我应该创建4个从同一个通道读取并顺序提供数字的goroutine吗?还是我应该为每个goroutine创建一个独立的通道?

还有一个问题,如果其中一个goroutine的测试失败,我该如何停止所有4个goroutine?我已经阅读了一些关于通道的文本,但是我无法将这些知识整合起来。

英文:

Suppose I have a simple loop which does sequential tests like this.

 for f := 1; f &lt;= 1000; f++ {
    		if doTest(f) {
              break
            }
  }

I loop through range of numbers and do a test for each number. If test fails for one number, I break and exit the main thread. Simple enough.

Now, how do correctly feed the test numbers in say four or several go routines. Basically, I want to test the numbers from 1 to 1000 in batches of 4 (or whatever number of go routines is).
Do I create 4 routines reading from one channel and feed the numbers sequentially into this channel? Or do I make 4 routines with an individual channel?

And another question. How do I stop all 4 routines if one of them fails the test? I've been reading some texts on channels but I cannot put the pieces together.

答案1

得分: 2

你可以创建一个生产者/消费者系统,代码如下:

func main() {
    ch := make(chan int)
    clients := 4
    // make it buffered, so all clients can fail without hanging
    notifyCh := make(chan struct{}, clients)
    go produce(100, ch, notifyCh)

    var wg sync.WaitGroup
    wg.Add(clients)
    for i := 0; i < clients; i++ {
        go func() {
            consumer(ch, notifyCh)
            wg.Done()
        }()
    }
    wg.Wait()

}

func consumer(in chan int, notifyCh chan struct{}) {
    fmt.Printf("Start consumer\n")
    for i := range in {
        <-time.After(100 * time.Millisecond)
        if i == 42 {
            fmt.Printf("%d fails\n", i)
            notifyCh <- struct{}{}
            return
        } else {
            fmt.Printf("%d\n", i)
        }

    }
    fmt.Printf("Consumer stopped working\n")
}

func produce(N int, out chan int, notifyCh chan struct{}) {
    for i := 0; i < N; i++ {
        select {
        case out <- i:
        case <-notifyCh:
            close(out)
            return
        }
    }
    close(out)
}

这个生产者将数字从0到99推送到通道,消费者在通道关闭之前一直消费。在main函数中,我们创建了4个客户端,并将它们添加到一个waitgroup中,以可靠地检查每个goroutine是否返回。
每个消费者都可以在notifyCh上发出信号,生产者停止工作并且不再生成更多的数字,因此所有消费者在处理完当前数字后返回。

还有一种方法是创建4个goroutine,等待它们全部返回,然后再启动下一个4个goroutine。但这样会增加等待的开销。

由于你提到了素数,这里有一个非常酷的素数筛选算法:https://golang.org/doc/play/sieve.go

英文:

You can create a producer/consumer system: https://play.golang.org/p/rks0gB3aDb

func main() {
	ch := make(chan int)
	clients := 4
	// make it buffered, so all clients can fail without hanging
	notifyCh := make(chan struct{}, clients)
	go produce(100, ch, notifyCh)

	var wg sync.WaitGroup
	wg.Add(clients)
	for i := 0; i &lt; clients; i++ {
		go func() {
			consumer(ch, notifyCh)
			wg.Done()
		}()
	}
	wg.Wait()

}

func consumer(in chan int, notifyCh chan struct{}) {
	fmt.Printf(&quot;Start consumer\n&quot;)
	for i := range in {
		&lt;-time.After(100 * time.Millisecond)
		if i == 42 {
			fmt.Printf(&quot;%d fails\n&quot;, i)
			notifyCh &lt;- struct{}{}
			return
		} else {
			fmt.Printf(&quot;%d\n&quot;, i)
		}

	}
	fmt.Printf(&quot;Consumer stopped working\n&quot;)
}

func produce(N int, out chan int, notifyCh chan struct{}) {
	for i := 0; i &lt; N; i++ {
		select {
		case out &lt;- i:
		case &lt;-notifyCh:
			close(out)
			return
		}
	}
	close(out)
}

The producer pushes numbers from 0 to 99 to the channel, the consumer consumes until the channel is closed. In main we create 4 clients and add them to a waitgroup to reliably check if every goroutine returned.
Every consumer can signal on the notifyCh, the producer stops working and no further numbers are generated, therefor all consumers return after their current number.

There's also an option to create 4 go routines, wait for all of them to return, start the next 4 go routines. But this adds quite an overhead on waiting.

Since you mentioned prime numbers, here's a really cool prime seive: https://golang.org/doc/play/sieve.go

答案2

得分: 0

你是要创建一个共享的通道还是每个例程一个通道取决于你的需求。

如果你只想把一些数字(或更一般的请求)放进去,而不关心哪个goroutine来处理,那当然最好是共享一个通道。但如果你想让前250个请求由goroutine1处理,那当然就不能共享一个通道了。

对于通道,最好的做法是将其用作输入或输出。发送方可以通过关闭通道来表示发送完成。关于这方面的好文章可以参考https://blog.golang.org/pipelines。

问题中没有提到的是,你是否还需要另一个通道(或多个通道)或其他通信原语来获取结果。在这里,通道比输入更有趣。

应该发送什么信息?可以在每次doTest之后发送一个布尔值,或者只需知道什么时候所有操作都完成(在这种情况下,不需要布尔值,只需关闭通道)。

如果你希望程序在第一次失败时停止。那么我建议使用带缓冲的共享通道来传递数字。不要忘记在所有数字都被传递完毕后关闭它。

还需要另一个非缓冲通道来让主线程知道测试是否完成。可以使用一个只放置失败测试的数字的通道,或者如果你还想要一个正面的结果,可以使用一个包含数字和结果的结构体的通道,或者使用从doTest返回的任何其他信息。

关于通道的非常好的文章还有http://dave.cheney.net/2014/03/19/channel-axioms。

你的四个goroutine中的每一个都可以报告一个失败(通过发送错误和关闭通道)。但问题是,当所有数字都通过并且输入通道被关闭时,goroutine应该做什么。关于这个问题,也有一篇很好的文章http://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/。

英文:

Whether you will create one channel common or a channel per routines depend on what you want.

If you want only put some numbers (or more general - requests) inside and you don't care which goroutine serve that, than of course is better to share a channel. In case when you want for example first 250 request to be served by goroutine1, than of course you cannot share a channel.

For channel is a good practice use it as input or output. And the simples thing how sender can sent, that he is finished is close the channel. Good article about that is https://blog.golang.org/pipelines

What is not mentiond in the question - is you need also another channel (or channels) or or any other communication primitive to get results. And here is the channel most interesting than to feeding.

What information should be sent - it should be sent, a bool after every doTest, or just know when everthing was done (it this case neither bool is not necessary just close a channel)?

If you prefer program at first fail. Than I would prefer use buffered shared channel to feed the numbers. Don't forget to close it, when all numbers will be feed.

And another unbuffered chan to let main thread know, that tests are done. It can be channel, there you only put the number, where test failed, or if you also want a positive result - channel of struct containing number and result, or any other informantion returned from doTest.

Very good article about channel is also http://dave.cheney.net/2014/03/19/channel-axioms

Each of your four goroutines can report a failure (by sending error and closing channel). But gotcha is what goroutines should do, when all numbers passed and feeding channel is closed. And about that is also nice article http://nathanleclaire.com/blog/2014/02/15/how-to-wait-for-all-goroutines-to-finish-executing-before-continuing/

huangapple
  • 本文由 发表于 2015年10月18日 17:10:27
  • 转载请务必保留本文链接:https://go.coder-hub.com/33196183.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定