英文:
creating a request from bufio.Reader
问题
我正在尝试实现一个批处理处理程序,它接受多部分混合数据。
我目前有一个相对简单的实现,如下所示。稍后我将尝试聚合响应并发送多部分响应。
我目前的问题是,我无法将各个部分的主体解析为新的请求。
func handleBatchPost(w http.ResponseWriter, r *http.Request) {
// 读取多部分主体
reader, err := r.MultipartReader()
if err != nil {
http.Error(w, fmt.Sprintf("无法读取多部分 %v\n", err), http.StatusBadRequest)
}
// 读取每个部分
for {
part, err := reader.NextPart()
if err == io.EOF {
break
} else if err != nil {
http.Error(w, fmt.Sprintf("无法读取下一个部分 %v\n", err), http.StatusBadRequest)
return
}
// 检查内容类型是否为http
if part.Header.Get("Content-Type") != "application/http" {
http.Error(w, fmt.Sprintf("部分的内容类型错误:%s\n", part.Header.Get("Content-Type")), http.StatusBadRequest)
return
}
// 将部分的主体解析为请求
req, err := http.ReadRequest(bufio.NewReader(part))
if err != nil {
http.Error(w, fmt.Sprintf("无法创建请求:%s\n", err), http.StatusBadRequest)
return
}
// 处理请求
router.ServeHTTP(w, req)
}
}
func handleItemPost(w http.ResponseWriter, r *http.Request) {
var item map[string]interface{}
if err := json.NewDecoder(r.Body).Decode(&item); err != nil {
http.Error(w, fmt.Sprintf("无法解码项目json:%v\n", err), http.StatusBadRequest)
return
}
w.Write([]byte(`{"success": true}`))
}
我从服务器收到了一个错误响应。似乎ReadRequest
只读取了头部(方法、URL等),而没有读取主体。
无法解码项目json:EOF
这是我发送的有效载荷。
POST /batch HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary
--boundary
Content-Type: application/http
Content-ID: <item1>
POST /items HTTP/1.1
Content-Type: application/json
{ "name": "batch1", "description": "batch1 description" }
--boundary
Content-Type: application/http
Content-ID: <item2>
POST /items HTTP/1.1
Content-Type: application/json
{ "name": "batch2", "description": "batch2 description" }
--boundary--
我在 Gmail API 文档中找到了这个模式 https://developers.google.com/gmail/api/guides/batch。
英文:
I am trying to implement a batch handler that accepts multipart mixed.
My currently somewhat naive implementation looks like the below. Later I will try to aggregate the responses and send a multipart response.
My current issue is that I am not able to parse the body of the individual parts into a new request.
func handleBatchPost(w http.ResponseWriter, r *http.Request) {
// read the multipart body
reader, err := r.MultipartReader()
if err != nil {
http.Error(w, fmt.Sprintf("could not read multipart %v\n", err), http.StatusBadRequest)
}
// read each part
for {
part, err := reader.NextPart()
if err == io.EOF {
break
} else if err != nil {
http.Error(w, fmt.Sprintf("could not read next part %v\n", err), http.StatusBadRequest)
return
}
// check if content type is http
if part.Header.Get("Content-Type") != "application/http" {
http.Error(w, fmt.Sprintf("part has wrong content type: %s\n", part.Header.Get("Content-Type")), http.StatusBadRequest)
return
}
// parse the body of the part into a request
req, err := http.ReadRequest(bufio.NewReader(part))
if err != nil {
http.Error(w, fmt.Sprintf("could not create request: %s\n", err), http.StatusBadRequest)
return
}
// handle the request
router.ServeHTTP(w, req)
}
}
func handleItemPost(w http.ResponseWriter, r *http.Request) {
var item map[string]interface{}
if err := json.NewDecoder(r.Body).Decode(&item); err != nil {
http.Error(w, fmt.Sprintf("could not decode item json: %v\n", err), http.StatusBadRequest)
return
}
w.Write([]byte(`{"success": true}`))
}
I am getting an error response from the server. It seems like ReadRequest
is not reading the body but only the headers (method, url, etc).
could not decode item json: EOF
This is the payload I am sending.
POST /batch HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary
--boundary
Content-Type: application/http
Content-ID: <item1>
POST /items HTTP/1.1
Content-Type: application/json
{ "name": "batch1", "description": "batch1 description" }
--boundary
Content-Type: application/http
Content-ID: <item2>
POST /items HTTP/1.1
Content-Type: application/json
{ "name": "batch2", "description": "batch2 description" }
--boundary--
I found this pattern on the gmail api docs https://developers.google.com/gmail/api/guides/batch.
答案1
得分: 1
主要问题是您的有效载荷没有为子请求指定Content-Length
头。如果缺少Content-Length
头,http.ReadRequest()
将假定没有主体,不会读取和呈现实际主体,这就是为什么会出现EOF错误的原因。
因此,首先提供缺失的Content-Length
头:
POST /batch HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary
--boundary
Content-Type: application/http
Content-ID: <item1>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch1", "description": "batch1 description" }
--boundary
Content-Type: application/http
Content-ID: <item2>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch2", "description": "batch2 description" }
--boundary--
这样应该可以工作,但请注意,由于您在同一个循环中处理部分,并在最后调用router.ServeHTTP(w, req)
,您正在重用w
写入器。这意味着什么?如果handleItemPost()
向输出写入任何内容,后续对handleItemPost()
的调用将无法撤消该内容。
例如,如果handleItemPost()
失败,它将以HTTP错误响应(这意味着设置响应状态并写入主体)。后续的handleItemPost()
无法再报告错误(标头已经提交),而且如果它报告成功,错误标头已经发送,只能将进一步的消息写入错误主体。
因此,例如,如果我们将handleItemPost()
修改为以下内容:
func handleItemPost(w http.ResponseWriter, r *http.Request) {
var item map[string]interface{}
if err := json.NewDecoder(r.Body).Decode(&item); err != nil {
fmt.Printf("JSON decode error: %v\n", err)
return
}
fmt.Printf("Success, item: %v\n", item)
}
并执行以下curl
命令:
curl localhost:8080/batch -X POST \
-H "Content-Type: multipart/mixed; boundary=boundary" \
-d '--boundary
Content-Type: application/http
Content-ID: <item1>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch1", "description": "batch1 description" }
--boundary
Content-Type: application/http
Content-ID: <item2>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch2", "description": "batch2 description" }
--boundary--'
我们将看到以下输出:
Success, item: map[description:batch1 description name:batch1]
Success, item: map[description:batch2 description name:batch2]
请注意,如果handleItemPost()
需要保持完全功能并可独立调用(以处理请求并生成响应),则不能对所有调用使用相同的http.ResponseWriter
。
在这种情况下,您可以为每个调用创建和使用单独的http.ResponseWriter
。标准库中有一个httptest.ResponseRecorder
类型,它实现了http.ResponseWriter
。它主要用于测试目的,但您也可以在这里使用它。它记录了编写的响应,因此您可以在调用之后检查它。
例如:
w2 := httptest.NewRecorder()
router.ServeHTTP(w2, req)
if w2.Code != http.StatusOK {
fmt.Printf("handleItemPost returned non-OK status: %v\n", w2.Code)
fmt.Printf("\terror body: %v\n", w2.Body.String())
}
使用您的原始请求运行此代码(未指定Content-Length
),输出将为:
handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF
handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF
但是,当您指定子请求的Content-Length
时,不会打印任何输出(错误)。
英文:
The main problem is that your payload does not specify Content-Length
header for the sub-requests. In case of a missing Content-Length
header, http.ReadRequest()
will assume no body, will not read and present the actual body, this is why you get EOF errors.
So first provide the missing Content-Length
headers:
POST /batch HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary
--boundary
Content-Type: application/http
Content-ID: <item1>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch1", "description": "batch1 description" }
--boundary
Content-Type: application/http
Content-ID: <item2>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch2", "description": "batch2 description" }
--boundary--
With this it should work, but note that since you are processing parts in the same loop, and calling router.ServeHTTP(w, req)
in the end, you're reusing the w
writer. What does this mean? If handleItemPost()
writes anything to the output, subsequent calls to handleItemPost()
can't take that back.
E.g. if a handleItemPost()
fails, it responds with an HTTP error (this implies setting response status and writing the body). A subsequent handleItemPost()
can't report an error again (headers are already committed), and also if it would report success, the error header is already sent and could only write further message to the error body.
So for example if we modify handleItemPost()
to this:
func handleItemPost(w http.ResponseWriter, r *http.Request) {
var item map[string]interface{}
if err := json.NewDecoder(r.Body).Decode(&item); err != nil {
fmt.Printf("JSON decode error: %v\n", err)
return
}
fmt.Printf("Success, item: %v\n", item)
}
And execute the following curl
command:
curl localhost:8080/batch -X POST \
-H "Content-Type: multipart/mixed; boundary=boundary" \
-d '--boundary
Content-Type: application/http
Content-ID: <item1>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch1", "description": "batch1 description" }
--boundary
Content-Type: application/http
Content-ID: <item2>
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch2", "description": "batch2 description" }
--boundary--'
We will see the following output:
Success, item: map[description:batch1 description name:batch1]
Success, item: map[description:batch2 description name:batch2]
Note that if handleItemPost()
needs to remain fully functional and callable on its own (to process the request and produce response), you can't use the same http.ResponseWriter
for all of its calls.
In this case you may create and use a separate http.ResponseWriter
for each of its invocation. The standard lib has an httptest.ResponseRecorder
type that implements http.ResponseWriter
. It's primarily for testing purposes, but you may use it here too. It records the written response, so you may inspect it after the call.
For example:
w2 := httptest.NewRecorder()
router.ServeHTTP(w2, req)
if w2.Code != http.StatusOK {
fmt.Printf("handleItemPost returned non-OK status: %v\n", w2.Code)
fmt.Printf("\terror body: %v\n", w2.Body.String())
}
Running this with your original request (without specifying Content-Length
), the output will be:
handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF
handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF
But when you specify the Content-Length
of the sub-requests, no output (error) is printed.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论