通过代理的HTTPS完全加密,包括SSL CONNECT。

huangapple go评论124阅读模式
英文:

HTTPS through proxy completely encrypted, including SSL CONNECT

问题

我正在尝试测试一个代理,该代理期望立即进行SSL握手,即使客户端将使用SSL CONNECT方法。问题是,我的Golang和Python测试代码似乎都有相同的缺陷。它们以明文连接到代理,然后发出CONNECT请求,但代理拒绝此请求,因为它期望进行SSL握手。

有人知道如何强制使用可用的标准库使任一技术使用SSL连接到代理吗?

谢谢。

以下是示例代码:

Python代码:

  1. #!/usr/bin/python
  2. try:
  3. import urllib2
  4. proxy = urllib2.ProxyHandler({'https': "myproxyip:6881"})
  5. opener = urllib2.build_opener(proxy)
  6. urllib2.install_opener(opener)
  7. print urllib2.urlopen('https://www.google.ca').read()
  8. except Exception as err:
  9. print err
  10. try:
  11. import httplib
  12. c = httplib.HTTPSConnection('myproxyip', 6881)
  13. c.set_tunnel('google.ca', 443)
  14. c.request('GET', '/')
  15. res = c.getresponse()
  16. print res.status, res.reason
  17. except Exception as err:
  18. print err

Golang代码:

  1. // vim: ft=go ts=4 sw=4 et ai:
  2. package main
  3. import (
  4. "net/http"
  5. "log"
  6. "io/ioutil"
  7. "time"
  8. "crypto/tls"
  9. "net/url"
  10. )
  11. var use_proxy = true
  12. var proxy = "https://myproxyip:6881"
  13. var req_url = "https://google.ca"
  14. var n_seconds time.Duration = 15
  15. var period = time.Second * n_seconds
  16. var n_threads = 50
  17. var thread_start_delay time.Duration = 1
  18. func http_fetch(req_url string) {
  19. tr := http.Transport {
  20. TLSClientConfig: &tls.Config {
  21. InsecureSkipVerify: true,
  22. },
  23. }
  24. proxy_url, err := url.Parse(proxy)
  25. if err != nil {
  26. log.Fatal(err)
  27. }
  28. if use_proxy {
  29. tr.Proxy = http.ProxyURL(proxy_url)
  30. }
  31. client := &http.Client{}
  32. client.Transport = &tr
  33. req, err := http.NewRequest("GET", req_url, nil)
  34. if err != nil {
  35. log.Fatal(err)
  36. }
  37. req.Header.Set("User-Agent", "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36")
  38. res, err := client.Do(req)
  39. if err != nil {
  40. log.Fatal(err)
  41. }
  42. body, err := ioutil.ReadAll(res.Body)
  43. defer res.Body.Close()
  44. document := string(body)
  45. print(document)
  46. }
  47. func main() {
  48. println("main: Starting", n_threads, "threads to hammer", req_url, "every", n_seconds, "seconds, ctrl-c to quit...")
  49. done := make(<-chan bool)
  50. for i:= 0; i < n_threads; i++ {
  51. go func(thread_id int) {
  52. println("thread", thread_id, ": starting periodic_hammer")
  53. for {
  54. println("thread", thread_id, ": fetching", req_url)
  55. http_fetch(req_url)
  56. println("thread", thread_id, ": done, sleeping for", n_seconds, "seconds")
  57. time.Sleep(period)
  58. }
  59. }(i+1)
  60. println("main: delaying", thread_start_delay, "seconds before starting next thread")
  61. time.Sleep(thread_start_delay * time.Second)
  62. }
  63. <-done
  64. }

希望对你有所帮助!

英文:

I am trying to test a proxy that is expecting an SSL handshake immediately, even if the client is going to use an SSL CONNECT method. The problem is that my Golang and Python test code both seem to have the same flaw. They connect to the proxy in the clear, and then issue a CONNECT, but the proxy rejects this because it is expecting an SSL handshake.

Does anyone know how to force either technology to use SSL to the proxy using available standard libraries?

Thanks.

Sample code follows:

  1. #!/usr/bin/python
  2. try:
  3. import urllib2
  4. proxy = urllib2.ProxyHandler({&#39;https&#39;: &quot;myproxyip:6881&quot;})
  5. opener = urllib2.build_opener(proxy)
  6. urllib2.install_opener(opener)
  7. print urllib2.urlopen(&#39;https://www.google.ca&#39;).read()
  8. except Exception as err:
  9. print err
  10. try:
  11. import httplib
  12. c = httplib.HTTPSConnection(&#39;myproxyip&#39;, 6881)
  13. c.set_tunnel(&#39;google.ca&#39;, 443)
  14. c.request(&#39;GET&#39;, &#39;/&#39;)
  15. res = c.getresponse()
  16. print res.status, res.reason
  17. except Exception as err:
  18. print err

and golang

  1. // vim: ft=go ts=4 sw=4 et ai:
  2. package main
  3. import (
  4. &quot;net/http&quot;
  5. &quot;log&quot;
  6. &quot;io/ioutil&quot;
  7. &quot;time&quot;
  8. &quot;crypto/tls&quot;
  9. &quot;net/url&quot;
  10. )
  11. var use_proxy = true
  12. var proxy = &quot;https://myproxyip:6881&quot;
  13. var req_url = &quot;https://google.ca&quot;
  14. var n_seconds time.Duration = 15
  15. var period = time.Second * n_seconds
  16. var n_threads = 50
  17. var thread_start_delay time.Duration = 1
  18. func http_fetch(req_url string) {
  19. tr := http.Transport {
  20. TLSClientConfig: &amp;tls.Config {
  21. InsecureSkipVerify: true,
  22. },
  23. }
  24. proxy_url, err := url.Parse(proxy)
  25. if err != nil {
  26. log.Fatal(err)
  27. }
  28. if use_proxy {
  29. tr.Proxy = http.ProxyURL(proxy_url)
  30. }
  31. client := &amp;http.Client{}
  32. client.Transport = &amp;tr
  33. req, err := http.NewRequest(&quot;GET&quot;, req_url, nil)
  34. if err != nil {
  35. log.Fatal(err)
  36. }
  37. req.Header.Set(&quot;User-Agent&quot;, &quot;Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36&quot;)
  38. res, err := client.Do(req)
  39. if err != nil {
  40. log.Fatal(err)
  41. }
  42. body, err := ioutil.ReadAll(res.Body)
  43. defer res.Body.Close()
  44. document := string(body)
  45. print(document)
  46. }
  47. func main() {
  48. println(&quot;main: Starting&quot;, n_threads, &quot;threads to hammer&quot;, req_url, &quot;every&quot;, n_seconds, &quot;seconds, ctrl-c to quit...&quot;)
  49. done := make(&lt;-chan bool)
  50. for i:= 0; i &lt; n_threads; i++ {
  51. go func(thread_id int) {
  52. println(&quot;thread&quot;, thread_id, &quot;: starting periodic_hammer&quot;)
  53. for {
  54. println(&quot;thread&quot;, thread_id, &quot;: fetching&quot;, req_url)
  55. http_fetch(req_url)
  56. println(&quot;thread&quot;, thread_id, &quot;: done, sleeping for&quot;, n_seconds, &quot;seconds&quot;)
  57. time.Sleep(period)
  58. }
  59. }(i+1)
  60. println(&quot;main: delaying&quot;, thread_start_delay, &quot;seconds before starting next thread&quot;)
  61. time.Sleep(thread_start_delay * time.Second)
  62. }
  63. &lt;-done
  64. }

答案1

得分: 1

使用https代理仅支持Go 1.10及以上版本。根据草案发布说明

> 在客户端方面,现在可以将HTTP代理(通常由ProxyFromEnvironment配置)指定为https:// URL,这意味着客户端在发出标准的代理HTTP请求之前通过HTTPS连接到代理。 (以前,HTTP代理URL必须以http://或socks5://开头。)

英文:

Using https proxies is only supported starting with Go 1.10. From the draft release notes:

> On the client side, an HTTP proxy (most commonly configured by
> ProxyFromEnvironment) can now be specified as an https:// URL, meaning
> that the client connects to the proxy over HTTPS before issuing a
> standard, proxied HTTP request. (Previously, HTTP proxy URLs were
> required to begin with http:// or socks5://.)

答案2

得分: 0

使用Python标准库以及您提供的ProxyHandler,可以通过以下方式解决此问题:

  1. import ssl
  2. import urllib2
  3. # 设置密码和SSL设置
  4. ctx = ssl.create_default_context()
  5. ctx.check_hostname = False
  6. ctx.verify_mode = ssl.CERT_NONE
  7. ctx.set_ciphers('HIGH:!DH:!aNULL')
  8. # 设置代理设置
  9. proxy = urllib2.ProxyHandler({'https': '<https代理主机>:443', 'http': '<http代理主机>:8080'})
  10. opener = urllib2.build_opener(urllib2.HTTPSHandler(context=ctx), proxy)
  11. urllib2.install_opener(opener)

请注意,上述代码中的<https代理主机><http代理主机>应替换为实际的代理主机地址。

英文:

A possible resolution to this issue using Python standard libraries in addition to the ProxyHandler you have:

  1. import ssl, urllib2
  2. # Set ciphers and ssl settings
  3. ctx = ssl.create_default_context()
  4. ctx.check_hostname = False
  5. ctx.verify_mode = ssl.CERT_NONE
  6. ctx.set_ciphers(&#39;HIGH:!DH:!aNULL&#39;)
  7. # Set proxy settings
  8. proxy = urllib2.ProxyHandler({&#39;https&#39;:&#39;&lt;https proxy host&gt;:443&#39;, &#39;http&#39;:&#39;&lt;http proxy host&gt;:8080&#39;})
  9. opener = urllib2.build_opener(urllib2.HTTPSHandler(context=ctx), proxy)
  10. urllib2.install_opener(opener)

答案3

得分: 0

最后我不得不自己做客户端。

  1. func http_fetch(req_host string) {
  2. buf := make([]byte, 1024)
  3. conf := &tls.Config {
  4. InsecureSkipVerify: true,
  5. }
  6. conn, err := tls.Dial("tcp", proxy_host, conf)
  7. if err != nil {
  8. //panic(err)
  9. failure()
  10. return
  11. }
  12. defer conn.Close()
  13. // SSL连接
  14. if _, err = conn.Write([]byte("CONNECT " + req_host + " HTTP/1.0\r\n\r\n")); err != nil {
  15. // FIXME: 需要调试日志记录器吗?
  16. //panic(err)
  17. failure()
  18. return
  19. }
  20. // 读取响应
  21. if _, err = conn.Read(buf); err != nil {
  22. //panic(err)
  23. failure()
  24. return
  25. }
  26. // 发送HTTP GET请求
  27. if _, err = conn.Write([]byte("GET / HTTP/1.0\r\n")); err != nil {
  28. //panic(err)
  29. failure()
  30. return
  31. }
  32. if _, err = conn.Write([]byte("User-Agent: golang\r\n\r\n")); err != nil {
  33. //panic(err)
  34. failure()
  35. return
  36. }
  37. // 读取响应
  38. if _, err = conn.Read(buf); err != nil {
  39. //panic(err)
  40. failure()
  41. return
  42. }
  43. success()
  44. }
英文:

In the end I had to do my own client, essentially.

  1. func http_fetch(req_host string) {
  2. buf := make([]byte, 1024)
  3. conf := &amp;tls.Config {
  4. InsecureSkipVerify: true,
  5. }
  6. conn, err := tls.Dial(&quot;tcp&quot;, proxy_host, conf)
  7. if err != nil {
  8. //panic(err)
  9. failure()
  10. return
  11. }
  12. defer conn.Close()
  13. // SSL CONNECT
  14. if _, err = conn.Write([]byte(&quot;CONNECT &quot; + req_host + &quot; HTTP/1.0\r\n\r\n&quot;)); err != nil {
  15. // FIXME: need debug logger?
  16. //panic(err)
  17. failure()
  18. return
  19. }
  20. // Read response
  21. if _, err = conn.Read(buf); err != nil {
  22. //panic(err)
  23. failure()
  24. return
  25. }
  26. // Send http GET
  27. if _, err = conn.Write([]byte(&quot;GET / HTTP/1.0\r\n&quot;)); err != nil {
  28. //panic(err)
  29. failure()
  30. return
  31. }
  32. if _, err = conn.Write([]byte(&quot;User-Agent: golang\r\n\r\n&quot;)); err != nil {
  33. //panic(err)
  34. failure()
  35. return
  36. }
  37. // Read response
  38. if _, err = conn.Read(buf); err != nil {
  39. //panic(err)
  40. failure()
  41. return
  42. }
  43. success()
  44. }

huangapple
  • 本文由 发表于 2017年7月11日 22:25:23
  • 转载请务必保留本文链接:https://go.coder-hub.com/45037328.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定