How to get an HTTP Proxy response Header in Golang built-in HTTP client?

huangapple go评论89阅读模式
英文:

How to get an HTTP Proxy response Header in Golang built-in HTTP client?

问题

我正在使用默认的"net/http" Golang库通过HTTP代理进行简单的HTTP GET请求,并希望读取第一个代理回复的内容(对于使用CONNECT方法的HTTP客户端请求)。在纯文本中,它看起来像这样:

HTTP/1.1 200 OK
Request-Uid: <some id>
<another header>: <another value>

Golang代码:

...
proxyUrlParsed, errUrl := url.Parse(proxyUrl)
tr := &http.Transport{
   Proxy:   http.ProxyURL(proxyUrlParsed),
}
client := &http.Client{
   Transport: tr,
}
request, errReq := http.NewRequest("GET", targetUrl, nil)
response, errDo := client.Do(request)
// Response contains HTTP headers from the reply from the target resource but not the intermediate proxy.

我部分解决了这个问题,使用了DialContext,但我需要实现一些我发现不太方便且成本较高的协议部分以供以后支持。所以有没有一种简单而巧妙的方法来解决这个问题?

英文:

I'm doing a simple HTTP GET request with the default "net/http" Golang lib via HTTP proxy and want to read the content of the first proxy reply(for the HTTP client req with the CONNECT method). In the plain text, it looks like
How to get an HTTP Proxy response Header in Golang built-in HTTP client?

HTTP/1.1 200 OK
Request-Uid: &lt;some id&gt;
&lt;another header&gt;: &lt;another value&gt;

Golang code:

...
proxyUrlParsed, errUrl := url.Parse(proxyUrl)
tr := &amp;http.Transport{
   Proxy:   http.ProxyURL(proxyUrlParsed),
}
client := &amp;http.Client{
   Transport: tr,
}
request, errReq := http.NewRequest(&quot;GET&quot;, targetUrl, nil)
response, errDo := client.Do(request)
// Response contains HTTP headers from the reply from the target resource but not the intermediate proxy.

I partially solved it with DialContext, but I were needed to impl some parts of the protocol that I found not so handy and costly for later support. So is there an easy and clever way to do it?

答案1

得分: 1

隧道代理

以使用curl客户端为例。在请求https时,使用CONNECT方法连接到隧道连接。获取的流内容是TLS加密内容,代理无法解密。

如果有TLS证书,可以尝试解析响应流。

wireshark捕获https请求时,需要在浏览器中配置一个参数。证书将保存在指定的文件中。

http_proxy=127.0.0.1:8021 https_proxy=127.0.0.1:8021 curl -v https://qq.com
http_proxy=127.0.0.1:8021 https_proxy=127.0.0.1:8021 curl -v https://qq.com
package main

import (
	"fmt"
	"io"
	"log"
	"net"
	"net/http"
	"net/http/httputil"
	"os"
)

func main() {
	proxy := func(w http.ResponseWriter, req *http.Request) {
		log.Println("proxy", req.Method, req.RequestURI)
		if req.URL.Host != "" {
			if req.Method == http.MethodConnect {
				// 隧道
				conn, err := net.Dial("tcp", req.URL.Host)
				if err != nil {
					w.WriteHeader(502)
					fmt.Fprint(w, err)
					return
				}

				client, _, err := w.(http.Hijacker).Hijack()
				if err != nil {
					w.WriteHeader(502)
					fmt.Fprint(w, err)
					conn.Close()
					return
				}
				client.Write([]byte("HTTP/1.0 200 OK\r\n\r\n"))

				hr, hw := io.Pipe()
				go func() {
					io.Copy(os.Stdout, hr)
					hr.Close()
				}()
				go func() {
					// 将响应打印到标准输出
					io.Copy(io.MultiWriter(client, hw), conn)
					client.Close()
					conn.Close()
					hw.Close()
				}()
				go func() {
					io.Copy(conn, client)
					client.Close()
					conn.Close()
				}()
				return
			}

			httputil.NewSingleHostReverseProxy(req.URL).ServeHTTP(w, req)
		}
	}
	http.ListenAndServe(":8021", http.HandlerFunc(proxy))
}

反向代理

使用/net/http/httputil.ReverseProxy代理请求,设置ModifyResponse字段作为响应钩子。

package main

import (
	"log"
	"net/http"
	"net/http/httputil"
	"net/url"
)

func main() {
	proxy := httputil.NewSingleHostReverseProxy(&url.URL{Scheme: "http", Host: "127.0.0.1:8020"})
	proxy.ModifyResponse = func(w *http.Response) error {
		w.Header.Add("Author", "eudore")
		log.Println(w.Request.Method, w.Request.RequestURI, w.Status)
		return nil
	}
	http.ListenAndServe(":8021", proxy)
}

curl请求:

[root@node1 ~]# curl -I  127.0.0.1:8020?222
HTTP/1.1 401 Unauthorized
Www-Authenticate: Basic
Date: Thu, 17 Nov 2022 01:34:06 GMT

[root@node1 ~]# curl -I  127.0.0.1:8021?222
HTTP/1.1 401 Unauthorized
Author: eudore
Date: Thu, 17 Nov 2022 01:34:07 GMT
Www-Authenticate: Basic
英文:

tunnel proxy

Take the use of the curl client as an example. When requesting https, use the CONNECT method to connect to the tunnel connection. The obtained stream content is TLS encrypted content, which cannot be decrypted by the proxy.

If have a tls certificate, can try to parse the response stream.

When wireshark captures https requests, a parameter needs to be configured in the browser. The certificate is saved in the specified file

http_proxy=127.0.0.1:8021 https_proxy=127.0.0.1:8021 curl -v https://qq.com
http_proxy=127.0.0.1:8021 https_proxy=127.0.0.1:8021 curl -v https://qq.com
package main

import (
	&quot;fmt&quot;
	&quot;io&quot;
	&quot;log&quot;
	&quot;net&quot;
	&quot;net/http&quot;
	&quot;net/http/httputil&quot;
	&quot;os&quot;
)

func main() {
	proxy := func(w http.ResponseWriter, req *http.Request) {
		log.Println(&quot;proxy&quot;, req.Method, req.RequestURI)
		if req.URL.Host != &quot;&quot; {
			if req.Method == http.MethodConnect {
				// tunnel
				conn, err := net.Dial(&quot;tcp&quot;, req.URL.Host)
				if err != nil {
					w.WriteHeader(502)
					fmt.Fprint(w, err)
					return
				}

				client, _, err := w.(http.Hijacker).Hijack()
				if err != nil {
					w.WriteHeader(502)
					fmt.Fprint(w, err)
					conn.Close()
					return
				}
				client.Write([]byte(&quot;HTTP/1.0 200 OK\r\n\r\n&quot;))

				hr, hw := io.Pipe()
				go func(){
					io.Copy(os.Stdout, hr)
					hr.Close()
				}()
				go func() {
					// print response to stdout
					io.Copy(io.MultiWriter(client, hw), conn)
					client.Close()
					conn.Close()
					hw.Close()
				}()
				go func() {
					io.Copy(conn, client)
					client.Close()
					conn.Close()
				}()
				return
			}

			httputil.NewSingleHostReverseProxy(req.URL).ServeHTTP(w, req)
		}
	}
	http.ListenAndServe(&quot;:8021&quot;, http.HandlerFunc(proxy))
}

reverse proxy

use /net/http/httputil.ReverseProxy proxy a request ,set ModifyResponse field is response hook.

package main

import (
	&quot;log&quot;
	&quot;net/http&quot;
	&quot;net/http/httputil&quot;
	&quot;net/url&quot;
)

func main() {
	proxy := httputil.NewSingleHostReverseProxy(&amp;url.URL{Scheme: &quot;http&quot;, Host: &quot;127.0.0.1:8020&quot;})
	proxy.ModifyResponse = func(w *http.Response) error {
		w.Header.Add(&quot;Author&quot;, &quot;eudore&quot;)
		log.Println(w.Request.Method, w.Request.RequestURI, w.Status)
		return nil
	}
	http.ListenAndServe(&quot;:8021&quot;, proxy)
}

curl request:

[root@node1 ~]# curl -I  127.0.0.1:8020?222
HTTP/1.1 401 Unauthorized
Www-Authenticate: Basic
Date: Thu, 17 Nov 2022 01:34:06 GMT

[root@node1 ~]# curl -I  127.0.0.1:8021?222
HTTP/1.1 401 Unauthorized
Author: eudore
Date: Thu, 17 Nov 2022 01:34:07 GMT
Www-Authenticate: Basic

huangapple
  • 本文由 发表于 2022年11月16日 23:33:54
  • 转载请务必保留本文链接:https://go.coder-hub.com/74463204.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定