英文:
Sending a `Reader` between two functions
问题
我正在尝试进行一个GET
请求,然后将获取到的内容发送到另一个站点的POST
请求中。以下代码可以正常工作:
func deliver(sourceURL *url.URL, destinationURL *url.URL) {
client := &http.Client{}
sourceResponse, err := http.Get(sourceURL.String())
if err == nil {
body, _ := ioutil.ReadAll(sourceResponse.Body)
client.Post(destinationURL.String(), "text/html", strings.NewReader(string(body)))
}
}
但是这种方式将整个响应读入内存中,我该如何将响应流式传输到POST
请求中呢?以下是我尝试的方式:
func deliver(sourceURL *url.URL, destinationURL *url.URL) {
client := &http.Client{}
sourceResponse, err := http.Get(sourceURL.String())
if err == nil {
client.Post(destinationURL.String(), "text/html", sourceResponse.Body)
}
}
我原以为这样可以工作,但实际上它只发送了一个空请求(没有正文)。为什么会这样?难道client.Post
函数不会尝试读取响应吗?
这里有一个在Go Playground上的示例:http://play.golang.org/p/_EO80aj6wr
将dataReader
而不是sourceResponse.Body
发送过去可以完美地工作,但直接发送sourceResponse.Body
却不行。
示例链接:
- http://localhost:3000/fetch?dest=http://requestb.in/177ji621&src=http://www.google.com
- http://requestb.in/177ji621?inspect
英文:
I'm trying to make a GET
request and then send the content I get to another site in a POST
request. The following code works fine:
func deliver(sourceURL *url.URL, destinationURL *url.URL) {
client := &http.Client{}
sourceResponse, err := http.Get(sourceURL.String())
if err == nil {
body, _ := ioutil.ReadAll(sourceResponse.Body)
client.Post(destinationURL.String(), "text/html", strings.NewReader(string(body)))
}
}
But this reads the entire response into memory - how to I stream the response into the POST request instead? Here's what I'm trying:
func deliver(sourceURL *url.URL, destinationURL *url.URL) {
client := &http.Client{}
sourceResponse, err := http.Get(sourceURL.String())
if err == nil {
client.Post(destinationURL.String(), "text/html", sourceResponse.Body)
}
}
I'd assume this would work, but it sends only an empty request (no body). Why is this? Doesn't the client.Post
function actually attempt to read the response?
Here's an example on the Go playground: http://play.golang.org/p/_EO80aj6wr
Sending the dataReader
instead of sourceResponse.Body
works perfectly, but sending the sourceResponse.Body
directly doesn't work.
Example Links:
http://localhost:3000/fetch?dest=http://requestb.in/177ji621&src=http://www.google.com
http://requestb.in/177ji621?inspect
答案1
得分: 2
我做了一个小测试程序来尝试一下:
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
)
var client = &http.Client{}
func httpd() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
log.Printf("%s Request from %s", r.Method, r.RemoteAddr)
if r.Method == "GET" {
fmt.Fprintf(w, "Here is a GET body")
return
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Fatalf("body read failed: %v", err)
}
log.Printf("Body received from %s: %q", r.Method, body)
})
log.Fatal(http.ListenAndServe(":8080", nil))
}
func deliver(sourceURL *url.URL, destinationURL *url.URL) {
sourceResponse, err := http.Get(sourceURL.String())
if err != nil {
log.Fatalf("post failed: %v", err)
}
postResponse, err := client.Post(destinationURL.String(), "text/html", sourceResponse.Body)
if err != nil {
log.Fatalf("post failed: %v", err)
}
postResponse.Body.Close()
sourceResponse.Body.Close()
}
func main() {
go httpd()
src, err := url.Parse("http://127.0.0.1:8080/")
if err != nil {
log.Fatalf("src parse failed: %v", err)
}
dst, err := url.Parse("http://127.0.0.1:8080/")
if err != nil {
log.Fatalf("dst parse failed: %v", err)
}
deliver(src, dst)
}
它正常工作,并产生以下输出:
2014/10/04 08:40:26 GET Request from 127.0.0.1:48910
2014/10/04 08:40:26 POST Request from 127.0.0.1:48911
2014/10/04 08:40:26 Body received from POST: "Here is a GET body"
所以你的代码问题可能会通过你没有检查的错误来揭示出来,最有可能是在 Post 中的错误。
另外请注意,如果可能的话,你希望重用 http.Client
,这样可以使用连接池。
更新:在查看问题的更新后,我使用 http://requestb.in/
运行了我的测试程序,它没有显示请求体。但是我看到了 Transfer-Encoding: chunked
,所以我怀疑 requestb.in
不支持分块传输编码。
如果你向请求提供了完整的请求体,那么 Go 会设置 Content-Length
头部,但是如果你提供了一个流,Go 就会使用分块传输编码。尽管分块传输编码是 HTTP/1.1 规范的一个非可选部分,但有很多服务器不正确地支持它。
英文:
I made a little test program to try this out
package main
import (
"fmt"
"io/ioutil"
"log"
"net/http"
"net/url"
)
var client = &http.Client{}
func httpd() {
http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
log.Printf("%s Request from %s", r.Method, r.RemoteAddr)
if r.Method == "GET" {
fmt.Fprintf(w, "Here is a GET body")
return
}
body, err := ioutil.ReadAll(r.Body)
if err != nil {
log.Fatalf("body read failed: %v", err)
}
log.Printf("Body received from %s: %q", r.Method, body)
})
log.Fatal(http.ListenAndServe(":8080", nil))
}
func deliver(sourceURL *url.URL, destinationURL *url.URL) {
sourceResponse, err := http.Get(sourceURL.String())
if err != nil {
log.Fatalf("post failed: %v", err)
}
postResponse, err := client.Post(destinationURL.String(), "text/html", sourceResponse.Body)
if err != nil {
log.Fatalf("post failed: %v", err)
}
postResponse.Body.Close()
sourceResponse.Body.Close()
}
func main() {
go httpd()
src, err := url.Parse("http://127.0.0.1:8080/")
if err != nil {
log.Fatalf("src parse failed: %v", err)
}
dst, err := url.Parse("http://127.0.0.1:8080/")
if err != nil {
log.Fatalf("dst parse failed: %v", err)
}
deliver(src, dst)
}
It worked fine, producing this output
2014/10/04 08:40:26 GET Request from 127.0.0.1:48910
2014/10/04 08:40:26 POST Request from 127.0.0.1:48911
2014/10/04 08:40:26 Body received from POST: "Here is a GET body"
So the problem with your code will probably be revealed by the errors you aren't checking - the one in the Post most likely.
Also note that you want to re-use http.Client
if you can, so you can get connection pooling.
Update after reviewing update to question.
I tried my test program with http://requestb.in/
and it doesn't show the body. However I do see this Transfer-Encoding: chunked
so my suspicion is that requestb.in
doesn't support chunked transfer encoding.
If you provide the entire body to the request then go sets the Content-Length
header, however if you provide a stream then go uses chunked transfer encoding. Even though chunked transfer encoding is a non optional part of the HTTP/1.1 spec there are quite a few servers which don't support it properly.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论