Trouble understanding how this rate limiter works

huangapple go评论93阅读模式
英文:

Trouble understanding how this rate limiter works

问题

我一直在尝试为我的应用程序制作一个速率限制器,并找到了这段代码。阅读后,我仍然有一些难以理解它确切功能的问题。

我目前的理解:

1)调用SetSmallRateLimit和SetLongRateLimit来初始化通道,并在goroutine中运行处理程序。

2)当调用requestAndUnmarshal时,checkRateLimiter向队列通道发送一个信号。

我不明白的地方:

1)RateLimitHandler在time.After(pertime)的持续时间内休眠,然后清除队列通道。不确定triggerWatcher和returnChan在做什么。

2)checkTimeTrigger - 不明白这个函数在做什么或其目的。

var (
    smallRateChan rateChan
    longRateChan  rateChan
)

type rateChan struct {
    RateQueue   chan bool
    TriggerChan chan bool
}

// 每10秒10个请求
func SetSmallRateLimit(numrequests int, pertime time.Duration) {
    smallRateChan = rateChan{
        RateQueue:   make(chan bool, numrequests),
        TriggerChan: make(chan bool),
    }
    go rateLimitHandler(smallRateChan, pertime)
}

// 每10分钟500个请求
func SetLongRateLimit(numrequests int, pertime time.Duration) {
    longRateChan = rateChan{
        RateQueue:   make(chan bool, numrequests),
        TriggerChan: make(chan bool),
    }
    go rateLimitHandler(longRateChan, pertime)
}

func rateLimitHandler(RateChan rateChan, pertime time.Duration) {
    returnChan := make(chan bool)
    go timeTriggerWatcher(RateChan.TriggerChan, returnChan)
    for {
        <-returnChan
        <-time.After(pertime)
        go timeTriggerWatcher(RateChan.TriggerChan, returnChan)
        length := len(RateChan.RateQueue)
        for i := 0; i < length; i++ {
            <-RateChan.RateQueue
        }
    }
}

func timeTriggerWatcher(timeTrigger chan bool, returnChan chan bool) {
    timeTrigger <- true
    returnChan <- true
}

func requestAndUnmarshal(requestURL string, v interface{}) (err error) {
    checkRateLimiter(smallRateChan)
    checkRateLimiter(longRateChan)
    resp, err := http.Get(requestURL)
    defer resp.Body.Close()
    if err != nil {
        return
    }
    checkTimeTrigger(smallRateChan)
    checkTimeTrigger(longRateChan)
    if resp.StatusCode != http.StatusOK {
        return RiotError{StatusCode: resp.StatusCode}
    }

    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return
    }

    err = json.Unmarshal(body, v)
    if err != nil {
        return
    }
    return
}

func checkRateLimiter(RateChan rateChan) {
    if RateChan.RateQueue != nil && RateChan.TriggerChan != nil {
        RateChan.RateQueue <- true
    }
}

func checkTimeTrigger(RateChan rateChan) {
    if RateChan.RateQueue != nil && RateChan.TriggerChan != nil {
        select {
        case <-RateChan.TriggerChan:
        default:
        }
    }
}
英文:

I've been trying to make a rate limiter for my application and came across this code. After reading it I still have some trouble understanding what it does exactly.

My current understanding:

  1. SetSmallRateLimit and SetLongRateLimit are called to initialize the channels and starts running the handlers in a goroutine.

  2. When requestAndUnmarshal is called, checkRateLimiter sends a signal to the queue channel.

What I don't understand:

  1. RateLimitHandler sleeps the duration of time.After(pertime) and afterwards clears the queue channel. Not sure what triggerWatcher and returnChan are doing.

  2. checkTimeTrigger - Don't understand what this function is doing or its purpose.

    var (
    smallRateChan rateChan
    longRateChan rateChan
    )

    type rateChan struct {
    RateQueue chan bool
    TriggerChan chan bool
    }

    //10 requests every 10 seconds
    func SetSmallRateLimit(numrequests int, pertime time.Duration) {
    smallRateChan = rateChan{
    RateQueue: make(chan bool, numrequests),
    TriggerChan: make(chan bool),
    }
    go rateLimitHandler(smallRateChan, pertime)
    }

    //500 requests every 10 minutes
    func SetLongRateLimit(numrequests int, pertime time.Duration) {
    longRateChan = rateChan{
    RateQueue: make(chan bool, numrequests),
    TriggerChan: make(chan bool),
    }
    go rateLimitHandler(longRateChan, pertime)
    }

    func rateLimitHandler(RateChan rateChan, pertime time.Duration) {
    returnChan := make(chan bool)
    go timeTriggerWatcher(RateChan.TriggerChan, returnChan)
    for {
    <-returnChan
    <-time.After(pertime)
    go timeTriggerWatcher(RateChan.TriggerChan, returnChan)
    length := len(RateChan.RateQueue)
    for i := 0; i < length; i++ {
    <-RateChan.RateQueue
    }
    }
    }

    func timeTriggerWatcher(timeTrigger chan bool, returnChan chan bool) {
    timeTrigger <- true
    returnChan <- true
    }

    func requestAndUnmarshal(requestURL string, v interface{}) (err error) {
    checkRateLimiter(smallRateChan)
    checkRateLimiter(longRateChan)
    resp, err := http.Get(requestURL)
    defer resp.Body.Close()
    if err != nil {
    return
    }
    checkTimeTrigger(smallRateChan)
    checkTimeTrigger(longRateChan)
    if resp.StatusCode != http.StatusOK {
    return RiotError{StatusCode: resp.StatusCode}
    }

     body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
    return
    }
    err = json.Unmarshal(body, v)
    if err != nil {
    return
    }
    return
    

    }

    func checkRateLimiter(RateChan rateChan) {
    if RateChan.RateQueue != nil && RateChan.TriggerChan != nil {
    RateChan.RateQueue <- true
    }
    }

    func checkTimeTrigger(RateChan rateChan) {
    if RateChan.RateQueue != nil && RateChan.TriggerChan != nil {
    select {
    case <-RateChan.TriggerChan:
    default:
    }
    }
    }

答案1

得分: 1

我不认为你应该使用这段代码来学习任何有用的东西。我不确定,但它似乎试图限制请求速率,但是它是错误的。它允许进行一定数量的请求,然后等待一段时间间隔。在时间间隔之后,它允许你再次发出请求。所有这些都是以非常复杂的方式完成的。

但它可能导致非常奇怪的情况。假设你每小时发出1个请求,你的限制是每20秒500个请求。那么这段代码将导致你在500小时后等待20秒,然后再次允许发出请求。

checkTimeTrigger 从 RateChan.TriggerChan 中删除一条消息(如果有的话),如果没有消息,则不执行任何操作并立即返回。

这段代码显然不符合 DRY 原则。如果你想限制请求速率,最好使用 https://godoc.org/golang.org/x/time/rate

英文:

I don't think you should use this code to learn anything useful. I'm not sure but it seems to try to limit the request rate but it's wrong. It allows to make certain amount of request and then wait for a time interval. After the time interval it allows you to make requests again. All of that is done in a very sophisticated way.

But it can lead to very strange scenarios. Let's say you make 1req/h and your limit is 500req/20sec. Then this code would cause you to wait 20sec after 500 hours and allow to make requests again.

checkTimeTrigger removes a message from RateChan.TriggerChan if it has any and does nothing if it has not and returns immediately.

This code isn't obviously DRY. Better use https://godoc.org/golang.org/x/time/rate is you want to limit your request rate.

huangapple
  • 本文由 发表于 2016年1月7日 17:53:23
  • 转载请务必保留本文链接:https://go.coder-hub.com/34652057.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定