英文:
Different Response From cURL and Golang POST - Can't Understand Why
问题
我正在尝试使用Go语言的http客户端从服务器获取响应。
我希望通过Go执行的请求与以下curl命令完全相同：
curl --data "fulladdress=22280+S+209th+Way%2C+Queen+Creek%2C+AZ+85142" 'http://www.homefacts.com/hfreport.html'
我已经编写了相应的Go代码，并尝试使用一个名为curl-to-go的好用的服务，该服务为上述curl请求生成了以下Go代码：
// Generated by curl-to-Go: https://mholt.github.io/curl-to-go
body := strings.NewReader(`fulladdress=22280+S+209th+Way%2C+Queen+Creek%2C+AZ+85142`)
req, err := http.NewRequest("POST", "http://www.homefacts.com/hfreport.html", body)
if err != nil {
// 处理错误
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := http.DefaultClient.Do(req)
if err != nil {
// 处理错误
}
defer resp.Body.Close()
问题是我在curl命令和Go代码之间得到了不同的响应。curl命令返回以下响应正文：
<html><head><meta http-equiv="refresh" content="0;url=http://www.homefacts.com/address/Arizona/Maricopa-County/Queen-Creek/85142/22280-S-209th-Way.html"/></head></html>
这是预期的结果。然而，Go代码返回了一个冗长的HTML，这不是预期的结果。
我尝试在curl命令中添加--verbose以复制所有头信息，所以我通过我的Go代码添加了以下头信息：
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.Header.Set("User-Agent", "curl/7.51.0")
req.Header.Set("Accept", "*/*")
req.Header.Set("Content-Length", "56")
但是仍然没有成功，Go代码的输出与curl命令的输出仍然不同。
有没有办法让Go代码获得与curl相同的响应？
英文:
I'm trying to get a response from a server using golang's http client.
The request I'm looking to perform via go should be identical to the following curl command:
curl --data "fulladdress=22280+S+209th+Way%2C+Queen+Creek%2C+AZ+85142" 'http://www.homefacts.com/hfreport.html'
I've coded the equivalent go code, and also tried using a nice service called curl-to-go, which generates the following go code for the above curl request:
// Generated by curl-to-Go: https://mholt.github.io/curl-to-go
body := strings.NewReader(`fulladdress=22280+S+209th+Way%2C+Queen+Creek%2C+AZ+85142`)
req, err := http.NewRequest("POST", "http://www.homefacts.com/hfreport.html", body)
if err != nil {
// handle err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
resp, err := http.DefaultClient.Do(req)
if err != nil {
// handle err
}
defer resp.Body.Close()
The problem is that I keep getting a different response between the curl command and the go code. The curl command returns this response body:
<html><head><meta http-equiv="refresh" content="0;url=http://www.homefacts.com/address/Arizona/Maricopa-County/Queen-Creek/85142/22280-S-209th-Way.html"/></head></html>
which is the expected result. However the go code returns a lengthly HTML which is not the expected result.
I've tried adding --verbose to the curl command to copy all it's headers, so I added the following headers via my go code:
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
req.Header.Set("User-Agent", "curl/7.51.0")
req.Header.Set("Accept", "*/*")
req.Header.Set("Content-Length", "56")
But still no joy, the output from the go code remains different than the curl one.
Any ideas on how to get the same curl response from go?
答案1
得分: 2
感谢 @u_mulder 指出了我正确的方向。似乎默认的 Go http 客户端会默认跟随重定向头，而 curl 不会。
以下是更新后的代码，可以在 Go 和 curl 之间生成相同的结果：
body := strings.NewReader(`fulladdress=22280+S+209th+Way%2C+Queen+Creek%2C+AZ+85142`)
req, err := http.NewRequest("POST", "http://www.homefacts.com/hfreport.html", body)
if err != nil {
// 处理错误
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
resp, err := client.Do(req)
if err != nil {
// 处理错误
}
defer resp.Body.Close()
希望对你有帮助！
英文:
Thanks @u_mulder for pointing me out in the right direction. It seems that the default go http client follows the redirect header by default, while curl does not.
Here is the updated code that generates the same results between go and curl:
body := strings.NewReader(`fulladdress=22280+S+209th+Way%2C+Queen+Creek%2C+AZ+85142`)
req, err := http.NewRequest("POST", "http://www.homefacts.com/hfreport.html", body)
if err != nil {
// handle err
}
req.Header.Set("Content-Type", "application/x-www-form-urlencoded")
client := &http.Client{
CheckRedirect: func(req *http.Request, via []*http.Request) error {
return http.ErrUseLastResponse
},
}
resp, err := client.Do(req)
if err != nil {
// handle err
}
defer resp.Body.Close()
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论