英文:
Is it possible to limit how many goroutines run per second?
问题
我有一个URL列表，我需要使用goroutine并发地发送HTTP请求。有没有办法检查和限制每秒发送的HTTP请求数量？
英文:
I have a list of URLs that I need to use goroutine to fire off HTTP requests concurrently. Is there anyway to check and limit how many of those HTTP requests are sent per second?
答案1
得分: 13
这是一个使用Go语言编写的简单版本，它是Leaky Bucket算法的一种改编，使用了通道和goroutine。在发出请求之前，从rate通道接收一个令牌将检查速率，并在速率限制器为空时阻塞。
// 创建一个带缓冲的通道。// 通道的容量是最大的突发请求数。rate := make(chan struct{}, 10)go func() {ticker := time.NewTicker(100 * time.Millisecond)for range ticker.C {rate <- struct{}{}}}()
由于一系列请求的时间超过了平均速率，最终会变成并发请求，因此您可能还需要限制并发性。您可以添加第二个通道作为信号量，在发出请求之前添加一个令牌，并在请求完成后将其删除。
// 将并发限制为5semaphore := make(chan struct{}, 5)// 在请求函数中semaphore <- struct{}{}defer func() {<-semaphore}()
这里有一个稍微完整一些的示例：
https://play.golang.org/p/ZrTPLcdeDF
英文:
A very simple version of this in Go would be an adaptation of a Leaky Bucket algorithm, using a channel and a goroutine. Receiving a token from the rate channel before making a request will check the rate and block if the rate limiter is empty.
// create a buffered channel.// The capacity of the channel is maximum burst that can be made.rate := make(chan struct{}, 10)go func() {ticker := time.NewTicker(100 * time.Millisecond)for range ticker.C {rate <- struct{}{}}}()
Since a series of requests that take longer than the average rate will end up being concurrent, you may need to limit concurrency too. You can add a second channel as a semaphore, adding a token to the semaphore before making a request, and removing it when it's complete.
// limit concurrency to 5semaphore := make(chan struct{}, 5)// in request functionsemaphore <- struct{}{}defer func() {<-semaphore}()
A slightly more complete example is here:
答案2
得分: -1
这是一个简单的包，可以帮助你限制同时运行的goroutine数量。请查看https://github.com/zenthangplus/goccm
示例：
package mainimport ("fmt""goccm""time")func main() {// 限制同时运行的goroutine数量为3个。c := goccm.New(3)for i := 1; i <= 10; i++ {// 在任何goroutine之前都必须调用此函数c.Wait()go func(i int) {fmt.Printf("Job %d is running\n", i)time.Sleep(2 * time.Second)// 当goroutine完成时必须调用此函数// 或者你可以在goroutine的顶部使用`defer c.Done()`c.Done()}(i)}// 在关闭主程序后，必须调用此函数以确保所有goroutine都已完成c.WaitAllDone()}
英文:
Here is the simple package to help you limit the number of goroutines that are allowed to run concurrently. Please check https://github.com/zenthangplus/goccm
Example:
package mainimport ("fmt""goccm""time")func main() {// Limit 3 goroutines to run concurrently.c := goccm.New(3)for i := 1; i <= 10; i++ {// This function have to call before any goroutinec.Wait()go func(i int) {fmt.Printf("Job %d is running\n", i)time.Sleep(2 * time.Second)// This function have to when a goroutine has finished// Or you can use `defer c.Done()` at the top of goroutine.c.Done()}(i)}// This function have to call to ensure all goroutines have finished// after close the main program.c.WaitAllDone()}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论