Go ignores HTTP_PROXY environment variable

user_1513087 0
Viewed 974

I read all over the internet, that Go reads HTTP_PROXY environment variable and set proxy for default client. However, it is not working for me and I don't know why.

I'm on Ubuntu 20.04, Go was 1.16, so I upgraded to 1.17 but it still the same.

I have the program below and I execute this in terminal: HTTP_PROXY="http://localhost:8000" go run req.go I see that first Println prints out correct value, but proxy is not used.

func main() {
    fmt.Println(os.Getenv("HTTP_PROXY"))
    client := &http.Client{}
    resp, err := client.Get("http://localhost:8090/vm/1")
    if err != nil {
        log.Fatal(err)
    }
    body, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(string(body))
}

If I modify the code, and set proxy explicitly, it works.

u, err := url.Parse("http://localhost:8000")
if err != nil {
    log.Fatal(err)
}
client := &http.Client{
    Transport: &http.Transport{Proxy: http.ProxyURL(u)},
}
go environment-variables
3 Answers

After digging in the source code I have found the main reason and also a workaround. It is commented behavior, but quite deep. Source opensource.google/x/net/http/httpproxy/proxy.go line 118 and on line 181 is exact if which is responsible.

The issue is, that HTTP_PROXY is ignored when the request has URL localhost or 127.x.x.x. But you can easily add custom URI to /etc/hosts or C:\Windows\System32\drivers\etc\hosts.

127.0.0.1    localserver.loc

Then all requests must go to localserver.loc:8090 not localhost:8090. And it will work like a charm.

user_15130870

I assume that you have a NO_PROXY variable set to "localhost", which is respected by the DefaultTransport.

When explicitely setting a Proxy with http.ProxyURL(), this proxy is used regardless of NO_PROXY.

user_39821070

For anyone else stumbling on this, be sure that you didn't initialize transport with an empty struct. Here's the DefaultTransport (note the Proxy value):

var DefaultTransport RoundTripper = &Transport{
    Proxy: ProxyFromEnvironment,
    DialContext: defaultTransportDialContext(&net.Dialer{
        Timeout:   30 * time.Second,
        KeepAlive: 30 * time.Second,
    }),
    ForceAttemptHTTP2:     true,
    MaxIdleConns:          100,
    IdleConnTimeout:       90 * time.Second,
    TLSHandshakeTimeout:   10 * time.Second,
    ExpectContinueTimeout: 1 * time.Second,
}

So if you were to say something like:

if client.Transport == nil {
    client.Transport = &http.Transport{}
}

the proxy wouldn't be used. The fix for me was:

if client.Transport == nil {
    client.Transport = http.DefaultTransport.(*http.Transport).Clone()
}
user_5962850
Related