Go ignores HTTP_PROXY environment variable

I read all over the internet, that Go reads HTTP_PROXY environment variable and set proxy for default client. However, it is not working for me and I don't know why.

I'm on Ubuntu 20.04, Go was 1.16, so I upgraded to 1.17 but it still the same.

I have the program below and I execute this in terminal: HTTP_PROXY="http://localhost:8000" go run req.go
I see that first Println prints out correct value, but proxy is not used.

func main() {
	fmt.Println(os.Getenv("HTTP_PROXY"))
	client := &http.Client{}
	resp, err := client.Get("http://localhost:8090/vm/1")
	if err != nil {
		log.Fatal(err)
	}
	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(body))
}

If I modify the code, and set proxy explicitly, it works.

u, err := url.Parse("http://localhost:8000")
if err != nil {
	log.Fatal(err)
}
client := &http.Client{
	Transport: &http.Transport{Proxy: http.ProxyURL(u)},
}

After digging in the source code I have found the main reason and also a workaround. It is commented behavior, but quite deep. Source opensource.google/x/net/http/httpproxy/proxy.go line 118 and on line 181 is exact if which is responsible.

The issue is, that HTTP_PROXY is ignored when the request has URL localhost or 127.x.x.x. But you can easily add custom URI to /etc/hosts or C:\Windows\System32\drivers\etc\hosts.

127.0.0.1    localserver.loc

Then all requests must go to localserver.loc:8090 not localhost:8090. And it will work like a charm.

I assume that you have a NO_PROXY variable set to "localhost", which is respected by the DefaultTransport.

When explicitely setting a Proxy with http.ProxyURL(), this proxy is used regardless of NO_PROXY.

For anyone else stumbling on this, be sure that you didn't initialize transport with an empty struct. Here's the DefaultTransport (note the Proxy value):

var DefaultTransport RoundTripper = &Transport{
	Proxy: ProxyFromEnvironment,
	DialContext: defaultTransportDialContext(&net.Dialer{
		Timeout:   30 * time.Second,
		KeepAlive: 30 * time.Second,
	}),
	ForceAttemptHTTP2:     true,
	MaxIdleConns:          100,
	IdleConnTimeout:       90 * time.Second,
	TLSHandshakeTimeout:   10 * time.Second,
	ExpectContinueTimeout: 1 * time.Second,
}

So if you were to say something like:

if client.Transport == nil {
	client.Transport = &http.Transport{}
}

the proxy wouldn't be used. The fix for me was:

if client.Transport == nil {
	client.Transport = http.DefaultTransport.(*http.Transport).Clone()
}