Go http, send incoming http.request to an other server using client.Do

huangapple go评论146阅读模式
英文:

Go http, send incoming http.request to an other server using client.Do

问题

这是我的用例

我们有一个名为"foobar"的服务,它有两个版本"legacy"和"version_2_of_doom"(都是用Go语言编写的)。

为了从"legacy"过渡到"version_2_of_doom",我们首先希望同时拥有这两个版本,并且在这个过渡期间,将POST请求(因为只有一个POST API调用)同时发送到两个版本。

我认为可以这样做:

  1. 修改"legacy"代码中的处理程序,以便在处理请求之初将请求复制到"version_2_of_doom"。
func(w http.ResponseWriter, req *http.Request) {
    req.URL.Host = "v2ofdoom.local:8081"
    req.Host = "v2ofdoom.local:8081"
    client := &http.Client{}
    client.Do(req)
    // legacy code
}

但是似乎并不像这样简单直接。

它会出现http: Request.RequestURI can't be set in client requests.的错误。

是否有一种常用的方法可以将一个http.Request传输到另一个服务器而不进行修改?

英文:

Here my use case

We have one services "foobar" which has two version legacy and version_2_of_doom (both in go)

In order to make the transition from legacy to version_2_of_doom , we would like in a first time, to have the two version alongside, and have the POST request (as there's only one POST api call in this ) received on both.

The way I see how to do it. Would be

  1. modifying the code of legacy at the beginning of the handler, in order to duplicate the request to version_2_of_doom

      func(w http.ResponseWriter, req *http.Request) {
          req.URL.Host = "v2ofdoom.local:8081"
          req.Host = "v2ofdoom.local:8081"
          client := &http.Client{}
          client.Do(req)
          // legacy code 
    

but it seems to not be as straightforward as this

it fails with http: Request.RequestURI can't be set in client requests.

Is there a well-known method to do this kind of action (i.e transfering without touching) a http.Request to an other server ?

答案1

得分: 37

你需要将想要的值复制到一个新的请求中。由于这与反向代理非常相似,你可能想看看"net/http/httputil"中的ReverseProxy是如何工作的。

创建一个新的请求,只复制你想要发送到下一个服务器的请求的部分。如果你打算在两个地方使用请求体,你还需要读取和缓冲请求体:

func handler(w http.ResponseWriter, req *http.Request) {
    // 如果我们想在这里读取请求体并发送请求,我们需要缓冲请求体
    body, err := ioutil.ReadAll(req.Body)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    // 如果需要将其解析为多部分,可以重新分配请求体
    req.Body = ioutil.NopCloser(bytes.NewReader(body))

    // 从客户端发送的原始RequestURI创建一个新的URL
    url := fmt.Sprintf("%s://%s%s", proxyScheme, proxyHost, req.RequestURI)

    proxyReq, err := http.NewRequest(req.Method, url, bytes.NewReader(body))

    // 我们可能想要过滤一些头部,否则我们可以直接使用浅拷贝
    // proxyReq.Header = req.Header
    proxyReq.Header = make(http.Header)
    for h, val := range req.Header {
        proxyReq.Header[h] = val
    }

    resp, err := httpClient.Do(proxyReq)
    if err != nil {
        http.Error(w, err.Error(), http.StatusBadGateway)
        return
    }
    defer resp.Body.Close()

    // 旧代码
}

以上是将请求的部分值复制到新请求的示例代码。

英文:

You need to copy the values you want into a new request. Since this is very similar to what a reverse proxy does, you may want to look at what "net/http/httputil" does for ReverseProxy.

Create a new request, and copy only the parts of the request you want to send to the next server. You will also need to read and buffer the request body if you intend to use it both places:

func handler(w http.ResponseWriter, req *http.Request) {
	// we need to buffer the body if we want to read it here and send it
	// in the request. 
	body, err := ioutil.ReadAll(req.Body)
	if err != nil {
		http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}

	// you can reassign the body if you need to parse it as multipart
	req.Body = ioutil.NopCloser(bytes.NewReader(body))

	// create a new url from the raw RequestURI sent by the client
	url := fmt.Sprintf("%s://%s%s", proxyScheme, proxyHost, req.RequestURI)

	proxyReq, err := http.NewRequest(req.Method, url, bytes.NewReader(body))

	// We may want to filter some headers, otherwise we could just use a shallow copy
	// proxyReq.Header = req.Header
	proxyReq.Header = make(http.Header)
	for h, val := range req.Header {
		proxyReq.Header[h] = val
	}

	resp, err := httpClient.Do(proxyReq)
	if err != nil {
		http.Error(w, err.Error(), http.StatusBadGateway)
		return
	}
	defer resp.Body.Close()

	// legacy code
}

答案2

得分: 12

根据我的经验,实现这个的最简单方法是创建一个新的请求对象,并将所需的所有请求属性复制到新的请求对象中:

func(rw http.ResponseWriter, req *http.Request) {
    url := req.URL
    url.Host = "v2ofdoom.local:8081"

    proxyReq, err := http.NewRequest(req.Method, url.String(), req.Body)
    if err != nil {
        // 处理错误
    }

    proxyReq.Header.Set("Host", req.Host)
    proxyReq.Header.Set("X-Forwarded-For", req.RemoteAddr)

    for header, values := range req.Header {
        for _, value := range values {
            proxyReq.Header.Add(header, value)
        }
    }

    client := &http.Client{}
    proxyRes, err := client.Do(proxyReq)
    
    // 其他操作...

}

这种方法的好处是不修改原始请求对象(也许你的处理程序函数或任何存在于堆栈中的中间件函数仍然需要原始对象)。

英文:

In my experience, the easiest way to achieve this was to simply create a new request and copy all request attributes that you need into the new request object:

func(rw http.ResponseWriter, req *http.Request) {
    url := req.URL
    url.Host = "v2ofdoom.local:8081"

    proxyReq, err := http.NewRequest(req.Method, url.String(), req.Body)
    if err != nil {
        // handle error
    }

    proxyReq.Header.Set("Host", req.Host)
    proxyReq.Header.Set("X-Forwarded-For", req.RemoteAddr)

    for header, values := range req.Header {
	    for _, value := range values {
		    proxyReq.Header.Add(header, value)
	    }
    }

    client := &http.Client{}
    proxyRes, err := client.Do(proxyReq)
    
    // and so on...

This approach has the benefit of not modifying the original request object (maybe your handler function or any middleware functions that are living in your stack still need the original object?).

答案3

得分: 3

func handler(w http.ResponseWriter, r *http.Request) {
	// 步骤1:重写URL
	URL, _ := url.Parse("https://full_generic_url:123/x/y")
	r.URL.Scheme = URL.Scheme
	r.URL.Host = URL.Host
	r.URL.Path = singleJoiningSlash(URL.Path, r.URL.Path)
	r.RequestURI = ""

	// 步骤2:调整Header
	r.Header.Set("X-Forwarded-For", r.RemoteAddr)

    // 注意:客户端应该在当前handler()之外创建
    client := &http.Client{} 
	// 步骤3:执行请求
    resp, err := client.Do(r)
	if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
		return
	}

	// 步骤4:将payload复制到响应写入器
	copyHeader(w.Header(), resp.Header)
	w.WriteHeader(resp.StatusCode)
	io.Copy(w, resp.Body)
	resp.Body.Close()
}

// copyHeader和singleJoiningSlash是从"/net/http/httputil/reverseproxy.go"复制过来的
func copyHeader(dst, src http.Header) {
	for k, vv := range src {
		for _, v := range vv {
			dst.Add(k, v)
		}
	}
}

func singleJoiningSlash(a, b string) string {
	aslash := strings.HasSuffix(a, "/")
	bslash := strings.HasPrefix(b, "/")
	switch {
	case aslash && bslash:
		return a + b[1:]
	case !aslash && !bslash:
		return a + "/" + b
	}
	return a + b
}
英文:

Using original request (copy or duplicate only if original request still need):

func handler(w http.ResponseWriter, r *http.Request) {
// Step 1: rewrite URL
URL, _ := url.Parse("https://full_generic_url:123/x/y")
r.URL.Scheme = URL.Scheme
r.URL.Host = URL.Host
r.URL.Path = singleJoiningSlash(URL.Path, r.URL.Path)
r.RequestURI = ""
// Step 2: adjust Header
r.Header.Set("X-Forwarded-For", r.RemoteAddr)
// note: client should be created outside the current handler()
client := &http.Client{} 
// Step 3: execute request
resp, err := client.Do(r)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// Step 4: copy payload to response writer
copyHeader(w.Header(), resp.Header)
w.WriteHeader(resp.StatusCode)
io.Copy(w, resp.Body)
resp.Body.Close()
}
// copyHeader and singleJoiningSlash are copy from "/net/http/httputil/reverseproxy.go"
func copyHeader(dst, src http.Header) {
for k, vv := range src {
for _, v := range vv {
dst.Add(k, v)
}
}
}
func singleJoiningSlash(a, b string) string {
aslash := strings.HasSuffix(a, "/")
bslash := strings.HasPrefix(b, "/")
switch {
case aslash && bslash:
return a + b[1:]
case !aslash && !bslash:
return a + "/" + b
}
return a + b
}

答案4

得分: 1

我看到了接受的答案,但我想说我不喜欢这个。我使用这段代码已经有几个月了,一直都能正常工作,但是在一段时间后,你会遇到破坏请求(在我的情况下是POST请求)。我更喜欢以下解决方案:

r.URL.Host = "example.com"
r.RequestURI = ""
client := &http.Client{}
delete(r.Header, "Accept-Encoding")
delete(r.Headers, "Content-Length")
resp, err := client.Do(r.WithContext(context.Background()))
if err != nil {
return nil, err
}
return resp, nil
英文:

I've seen the accepted anwser, but I would like to say that I dont like this. I've used this code for months with it working, but after some time you encounter requests that break (POST requests in my case). My preferred solution is the following:

r.URL.Host = "example.com"
r.RequestURI = ""
client := &http.Client{}
delete(r.Header, "Accept-Encoding")
delete(r.Headers, "Content-Length")
resp, err := client.Do(r.WithContext(context.Background())
if err != nil {
return nil, err
}
return resp, nil

huangapple
  • 本文由 发表于 2016年1月11日 22:40:21
  • 转载请务必保留本文链接:https://go.coder-hub.com/34724160.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定