Google App Engine Go – Reddit API 返回错误 429

huangapple go评论83阅读模式
英文:

Google App Engine Go - Reddit API returns error 429

问题

我正在尝试在Google App Engine中使用Golang创建Reddit API。我的代码:

package RedditAPI

import(
    "appengine"
    "encoding/json"
    "io/ioutil"
    "net/http"
    "appengine/urlfetch"
    "time"
    "net/url"
)

func GetTopSubmissions(c appengine.Context){
    one, two:=Call(c, "http://www.reddit.com/r/Bitcoin/top.json", "POST", nil);
    c.Infof("%v, %v", one, two);
}


func Call(c appengine.Context, address string, requestType string, values url.Values)(map[string]interface{}, error){
    req, err:=http.NewRequest("GET", address, nil)
    if err!=nil{
        c.Infof("Request: %v", err)
        return nil, err
    }
    
    req.Header.Add("User-Agent", "This is a very creative name for a Reddit bot v1.0 by /u/username")
    c.Infof("%v", req.Header.Get("User-Agent"))
    c.Infof("%v", req)
    c.Infof("%v", req.UserAgent())
    
    duration, err:= time.ParseDuration("60s")
    tr := &urlfetch.Transport{Context: c, Deadline: duration}
    
    resp, err:=tr.RoundTrip(req)
    if err != nil {
        c.Infof("Post: %v", err)
        return nil, err
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        c.Infof("ReadAll: %v", err)
        return nil, err
    }
    result := make(map[string]interface{})
    err = json.Unmarshal(body, &result)
    if err != nil {
        c.Infof("Unmarshal: %v", err)
        c.Infof("%s", body)
        return nil, err
    }
    return result, nil
}

返回:

2013/05/23 03:00:34 Unsolicited response received on idle HTTP channel starting with "H"; err=<nil>
2013/05/23 03:01:42 INFO: &{GET http://www.reddit.com/r/Bitcoin/top.json HTTP/1.1 1 1 map[User-Agent:[This is a very creative name for a Reddit bot v1.0 by /u/username]] <nil> 0 [] false www.reddit.com map[] <nil> map[]   <nil>}
2013/05/23 03:01:42 INFO: This is a very creative name for a Reddit bot v1.0 by /u/username
2013/05/23 03:01:42 INFO: map[error:429], <nil>
INFO     2013-05-23 03:01:42,720 server.py:584] default: "GET / HTTP/1.1" 200 81

这个错误的原因可能是什么?

英文:

I am trying to create a Reddit API in Golang in Google App Engine. My code:

package RedditAPI

import(
	&quot;appengine&quot;
	&quot;encoding/json&quot;
	&quot;io/ioutil&quot;
	&quot;net/http&quot;
    &quot;appengine/urlfetch&quot;
    &quot;time&quot;
	&quot;net/url&quot;
)

func GetTopSubmissions(c appengine.Context){
	one, two:=Call(c, &quot;http://www.reddit.com/r/Bitcoin/top.json&quot;, &quot;POST&quot;, nil);
	c.Infof(&quot;%v, %v&quot;, one, two);
}


func Call(c appengine.Context, address string, requestType string, values url.Values)(map[string]interface{}, error){
    req, err:=http.NewRequest(&quot;GET&quot;, address, nil)
    if err!=nil{
        c.Infof(&quot;Request: %v&quot;, err)
    	return nil, err
    }
    
    req.Header.Add(&quot;User-Agent&quot;, &quot;This is a very creative name for a Reddit bot v1.0 by /u/username&quot;)
    c.Infof(&quot;%v&quot;, req.Header.Get(&quot;User-Agent&quot;))
    c.Infof(&quot;%v&quot;, req)
    c.Infof(&quot;%v&quot;, req.UserAgent())
    
    duration, err:= time.ParseDuration(&quot;60s&quot;)
    tr := &amp;urlfetch.Transport{Context: c, Deadline: duration}
    
    resp, err:=tr.RoundTrip(req)
    if err != nil {
        c.Infof(&quot;Post: %v&quot;, err)
    	return nil, err
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        c.Infof(&quot;ReadAll: %v&quot;, err)
    	return nil, err
    }
    result := make(map[string]interface{})
    err = json.Unmarshal(body, &amp;result)
    if err != nil {
        c.Infof(&quot;Unmarshal: %v&quot;, err)
        c.Infof(&quot;%s&quot;, body)
    	return nil, err
    }
    return result, nil
}

Returns

2013/05/23 03:00:34 Unsolicited response received on idle HTTP channel starting with &quot;H&quot;; err=&lt;nil&gt;
2013/05/23 03:01:42 INFO: &amp;{GET http://www.reddit.com/r/Bitcoin/top.json HTTP/1.1 1 1 map[User-Agent:[This is a very creative name for a Reddit bot v1.0 by /u/username]] &lt;nil&gt; 0 [] false www.reddit.com map[] &lt;nil&gt; map[]   &lt;nil&gt;}
2013/05/23 03:01:42 INFO: This is a very creative name for a Reddit bot v1.0 by /u/username
2013/05/23 03:01:42 INFO: map[error:429], &lt;nil&gt;
INFO     2013-05-23 03:01:42,720 server.py:584] default: &quot;GET / HTTP/1.1&quot; 200 81

What can be the cause of this error?

答案1

得分: 1

HTTP错误429是“请求过多”:https://www.rfc-editor.org/rfc/rfc6585#section-4

在https://github.com/reddit/reddit/wiki/API上,Reddit说“每分钟不要超过30个请求。这样可以允许一些突发性的请求,但要保持合理。平均而言,我们应该每两秒钟收到不超过一个请求。”他们还说,“不要在30秒内多次访问同一页。”

如果您在每次用户访问您的应用程序时请求相同的资源,您可以使用GAE的内存缓存支持。如果您正在请求一堆资源,Reddit维基页面提到了“同时请求多个资源”,所以可能有一些批量请求的功能(不确定,对API不熟悉)。如果您正在执行定期作业,可以减少其频率。

无论如何,剩下的谜团更多是Reddit API的问题,而不是Go的问题。matthewbauer在(https://github.com/jzelinskie/reddit)中发表评论的存储库或Reddit API文档可能会有更多信息。

英文:

HTTP error 429 is 'too many requests': https://www.rfc-editor.org/rfc/rfc6585#section-4

At https://github.com/reddit/reddit/wiki/API, Reddit says "Make no more than thirty requests per minute. This allows some burstiness to your requests, but keep it sane. On average, we should see no more than one request every two seconds from you." They also say, "Don't hit the same page more than once per 30 seconds."

If you're requesting the same resource on every user hit to your app, you could use GAE's memcache support. If you're requesting a bunch of resources, the Reddit wiki page mentions "[r]equests for multiple resources at a time", so there may be some facility for batching requests (not sure, not versed in the API). If you're doing a cronjob, you could run it less frequently.

In any case, the remaining mystery is more of a Reddit API question than a Go question. The repo matthewbauer commented with (https://github.com/jzelinskie/reddit) or the Reddit API docs may have more.

huangapple
  • 本文由 发表于 2013年5月23日 18:06:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/16711070.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定