一个使用Go语言编写的HTTP代理

huangapple go评论76阅读模式
英文:

An HTTP proxy in go

问题

我正在使用Go编写一个简单的代理,将HTTP请求转发到一个或多个后端服务器。目前我仍然只使用一个后端服务器,但性能不好,我确定我做错了什么。可能与如何将HTTP请求发送到另一个服务器有关:如果我注释掉对send()的调用,那么服务器会飞快地运行,每秒处理超过14,000个请求。而调用send()后,性能下降到不到1,000个请求每秒,并且随着时间的推移性能还会进一步下降。这是在MacBook Pro上测试的结果。

这段代码基于简单的代码示例;我创建了client并按照文档中的建议进行了重用。使用Apache的ab工具进行测试:

$ ab -n 10000 -c 10 -k "http://127.0.0.1:8080/test"

在端口55455上运行的后端服务器在实验之间不会更改;可以使用Apache或nginx。我的自定义Web服务器在直接测量时每秒处理超过7,000个请求:

$ ab -n 10000 -c 10 -k "http://127.0.0.1:55455/test"

我希望代理版本的性能与非代理版本一样好,并且能够持续保持性能。

以下是完整的示例代码:

package main

import (
        "fmt"
        "log"
        "net/http"
)

func main() {

        tr := &http.Transport{
                DisableCompression: true,
                DisableKeepAlives: false,
        }
        client := &http.Client{Transport: tr}

        http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

                send(r, client)
                fmt.Fprintf(w, "OK")
        })
        log.Fatal(http.ListenAndServe(":8080", nil))
}

func send(r *http.Request, client *http.Client) int {

        req, err := http.NewRequest("GET", "http://localhost:55455" + r.URL.Path, nil)
        if err != nil {
                log.Fatal(err)
                return 0
        }       
        resp, err := client.Do(req)
        if err != nil {
                log.Fatal(err)
                return 0
        }       
        if resp == nil {
                return 0
        }       
        return 1
}       

最终,该代码应该将请求发送到多个服务器并处理它们的响应,返回一个包含结果的整数。但我在这个步骤上遇到了困难,只是在进行调用时卡住了。

我到底做错了什么?

英文:

I'm writing a simple proxy in go that relays an HTTP request to one or more backend servers. Right now I'm still using only one backend server, but performance is not good and I'm sure I am doing something wrong. Probably related to how I send the HTTP request to another server: if I comment the call to send() then the server goes blazing fast, yielding more than 14 krps. While with the call to send() performance drops to less than 1 krps and drops even lower with time. This on a MacBook Pro.

The code is based on trivial code samples; I have created the client and reused it following the recommendation in the docs. Tests are done with Apache ab:

$ ab -n 10000 -c 10 -k "http://127.0.0.1:8080/test"

The backend server running on port 55455 does not change between experiments; Apache or nginx can be used. My custom web server yields more than 7 krps when measured directly, without proxy:

$ ab -n 10000 -c 10 -k "http://127.0.0.1:55455/test"

I would expect the proxied version to behave just as well as the non-proxied version, and sustain the performance over time.

The complete sample code follows.

package main

import (
        "fmt"
        "log"
        "net/http"
        )

func main() {

        tr := &http.Transport{
                DisableCompression: true,
                DisableKeepAlives: false,
        }
        client := &http.Client{Transport: tr}

        http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {

                send(r, client)
                fmt.Fprintf(w, "OK")
        })
        log.Fatal(http.ListenAndServe(":8080", nil))
}

func send(r *http.Request, client *http.Client) int {

        req, err := http.NewRequest("GET", "http://localhost:55455" + r.URL.Path, nil)
        if err != nil {
                log.Fatal(err)
                return 0
        }       
        resp, err := client.Do(req)
        if err != nil {
                log.Fatal(err)
                return 0
        }       
        if resp == nil {
                return 0
        }       
        return 1
}       

Eventually the code should send the request to multiple servers and process their answers, returning an int with the result. But I'm stuck at this step of just making the call.

What am I doing horribly wrong?

答案1

得分: 7

根据评论的建议,你应该返回(并处理)类型错误而不是整数,并且重申一下,不要使用AB。我注意到最重要的是:

  1. 你应该在你的Transport中设置MaxIdleConnsPerHost。这表示即使连接当前没有任务,也会保持(keep-alive)多少个连接。
  2. 为了将响应返回给连接池,你必须读取并关闭响应的主体。
...
resp, err := client.Do(req)
if err != nil {
    return err
}
responseBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
    return err
}
defer resp.Body.Close()
...
英文:

As the comment suggests, you should be returning (and dealing with) type error instead of ints, and to reiterate, don't use AB. The biggest thing that stands out to me is

  1. You should set the MaxIdleConnsPerHost in your Transport. This represents how many connections will persist (keep-alive) even if they have nothing to do at the moment.
  2. You have to read and close the body of the response in order for it to be returned to the pool.

...
resp, err := client.Do(req)
if err != nil {
    return err
}
responseBody, err := ioutil.ReadAll(resp.Body)
if err != nil {
    return err
}
defer resp.Body.Close()
...

huangapple
  • 本文由 发表于 2014年8月25日 08:01:20
  • 转载请务必保留本文链接:https://go.coder-hub.com/25477442.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定