从bufio.Reader创建一个请求。

huangapple go评论87阅读模式
英文:

creating a request from bufio.Reader

问题

我正在尝试实现一个批处理处理程序,它接受多部分混合数据。

我目前有一个相对简单的实现,如下所示。稍后我将尝试聚合响应并发送多部分响应。

我目前的问题是,我无法将各个部分的主体解析为新的请求。

func handleBatchPost(w http.ResponseWriter, r *http.Request) {
  // 读取多部分主体
  reader, err := r.MultipartReader()
  if err != nil {
    http.Error(w, fmt.Sprintf("无法读取多部分 %v\n", err), http.StatusBadRequest)
  }

  // 读取每个部分
  for {
    part, err := reader.NextPart()
    if err == io.EOF {
      break
    } else if err != nil {
      http.Error(w, fmt.Sprintf("无法读取下一个部分 %v\n", err), http.StatusBadRequest)
      return
    }

    // 检查内容类型是否为http
    if part.Header.Get("Content-Type") != "application/http" {
      http.Error(w, fmt.Sprintf("部分的内容类型错误:%s\n", part.Header.Get("Content-Type")), http.StatusBadRequest)
      return
    }

    // 将部分的主体解析为请求
    req, err := http.ReadRequest(bufio.NewReader(part))
    if err != nil {
      http.Error(w, fmt.Sprintf("无法创建请求:%s\n", err), http.StatusBadRequest)
      return
    }

    // 处理请求
    router.ServeHTTP(w, req)
  }
}

func handleItemPost(w http.ResponseWriter, r *http.Request) {
  var item map[string]interface{}
  if err := json.NewDecoder(r.Body).Decode(&item); err != nil {
    http.Error(w, fmt.Sprintf("无法解码项目json:%v\n", err), http.StatusBadRequest)
    return
  }
  w.Write([]byte(`{"success": true}`))
}

我从服务器收到了一个错误响应。似乎ReadRequest只读取了头部(方法、URL等),而没有读取主体。

无法解码项目json:EOF

这是我发送的有效载荷。

POST /batch  HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary

--boundary
Content-Type: application/http
Content-ID: <item1>

POST /items HTTP/1.1
Content-Type: application/json

{ "name": "batch1", "description": "batch1 description" }

--boundary
Content-Type: application/http
Content-ID: <item2>

POST /items HTTP/1.1
Content-Type: application/json

{ "name": "batch2", "description": "batch2 description" }

--boundary--

我在 Gmail API 文档中找到了这个模式 https://developers.google.com/gmail/api/guides/batch

英文:

I am trying to implement a batch handler that accepts multipart mixed.

My currently somewhat naive implementation looks like the below. Later I will try to aggregate the responses and send a multipart response.

My current issue is that I am not able to parse the body of the individual parts into a new request.

func handleBatchPost(w http.ResponseWriter, r *http.Request) {
  // read the multipart body
  reader, err := r.MultipartReader()
  if err != nil {
    http.Error(w, fmt.Sprintf(&quot;could not read multipart %v\n&quot;, err), http.StatusBadRequest)
  }

  // read each part
  for {
    part, err := reader.NextPart()
    if err == io.EOF {
      break
    } else if err != nil {
      http.Error(w, fmt.Sprintf(&quot;could not read next part %v\n&quot;, err), http.StatusBadRequest)
      return
    }

    // check if content type is http
    if part.Header.Get(&quot;Content-Type&quot;) != &quot;application/http&quot; {
      http.Error(w, fmt.Sprintf(&quot;part has wrong content type: %s\n&quot;, part.Header.Get(&quot;Content-Type&quot;)), http.StatusBadRequest)
      return
    }

    // parse the body of the part into a request
    req, err := http.ReadRequest(bufio.NewReader(part))
    if err != nil {
      http.Error(w, fmt.Sprintf(&quot;could not create request: %s\n&quot;, err), http.StatusBadRequest)
      return
    }

    // handle the request
    router.ServeHTTP(w, req)
  }
}

func handleItemPost(w http.ResponseWriter, r *http.Request) {
  var item map[string]interface{}
  if err := json.NewDecoder(r.Body).Decode(&amp;item); err != nil {
    http.Error(w, fmt.Sprintf(&quot;could not decode item json: %v\n&quot;, err), http.StatusBadRequest)
    return
  }
  w.Write([]byte(`{&quot;success&quot;: true}`))
}

I am getting an error response from the server. It seems like ReadRequest is not reading the body but only the headers (method, url, etc).

could not decode item json: EOF

This is the payload I am sending.

POST /batch  HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary

--boundary
Content-Type: application/http
Content-ID: &lt;item1&gt;

POST /items HTTP/1.1
Content-Type: application/json

{ &quot;name&quot;: &quot;batch1&quot;, &quot;description&quot;: &quot;batch1 description&quot; }

--boundary
Content-Type: application/http
Content-ID: &lt;item2&gt;

POST /items HTTP/1.1
Content-Type: application/json

{ &quot;name&quot;: &quot;batch2&quot;, &quot;description&quot;: &quot;batch2 description&quot; }

--boundary--

I found this pattern on the gmail api docs https://developers.google.com/gmail/api/guides/batch.

答案1

得分: 1

主要问题是您的有效载荷没有为子请求指定Content-Length头。如果缺少Content-Length头,http.ReadRequest()将假定没有主体,不会读取和呈现实际主体,这就是为什么会出现EOF错误的原因。

因此,首先提供缺失的Content-Length头:

POST /batch  HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary
--boundary
Content-Type: application/http
Content-ID: &lt;item1&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ &quot;name&quot;: &quot;batch1&quot;, &quot;description&quot;: &quot;batch1 description&quot; }
--boundary
Content-Type: application/http
Content-ID: &lt;item2&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ &quot;name&quot;: &quot;batch2&quot;, &quot;description&quot;: &quot;batch2 description&quot; }
--boundary--

这样应该可以工作,但请注意,由于您在同一个循环中处理部分,并在最后调用router.ServeHTTP(w, req),您正在重用w写入器。这意味着什么?如果handleItemPost()向输出写入任何内容,后续对handleItemPost()的调用将无法撤消该内容。

例如,如果handleItemPost()失败,它将以HTTP错误响应(这意味着设置响应状态并写入主体)。后续的handleItemPost()无法再报告错误(标头已经提交),而且如果它报告成功,错误标头已经发送,只能将进一步的消息写入错误主体。

因此,例如,如果我们将handleItemPost()修改为以下内容:

func handleItemPost(w http.ResponseWriter, r *http.Request) {
var item map[string]interface{}
if err := json.NewDecoder(r.Body).Decode(&amp;item); err != nil {
fmt.Printf("JSON decode error: %v\n", err)
return
}
fmt.Printf("Success, item: %v\n", item)
}

并执行以下curl命令:

curl localhost:8080/batch -X POST \
-H "Content-Type: multipart/mixed; boundary=boundary" \
-d '--boundary
Content-Type: application/http
Content-ID: &lt;item1&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch1", "description": "batch1 description" }
--boundary
Content-Type: application/http
Content-ID: &lt;item2&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ "name": "batch2", "description": "batch2 description" }
--boundary--'

我们将看到以下输出:

Success, item: map[description:batch1 description name:batch1]
Success, item: map[description:batch2 description name:batch2]

请注意,如果handleItemPost()需要保持完全功能并可独立调用(以处理请求并生成响应),则不能对所有调用使用相同的http.ResponseWriter

在这种情况下,您可以为每个调用创建和使用单独的http.ResponseWriter。标准库中有一个httptest.ResponseRecorder类型,它实现了http.ResponseWriter。它主要用于测试目的,但您也可以在这里使用它。它记录了编写的响应,因此您可以在调用之后检查它。

例如:

w2 := httptest.NewRecorder()
router.ServeHTTP(w2, req)
if w2.Code != http.StatusOK {
fmt.Printf("handleItemPost returned non-OK status: %v\n", w2.Code)
fmt.Printf("\terror body: %v\n", w2.Body.String())
}

使用您的原始请求运行此代码(未指定Content-Length),输出将为:

handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF
handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF

但是,当您指定子请求的Content-Length时,不会打印任何输出(错误)。

英文:

The main problem is that your payload does not specify Content-Length header for the sub-requests. In case of a missing Content-Length header, http.ReadRequest() will assume no body, will not read and present the actual body, this is why you get EOF errors.

So first provide the missing Content-Length headers:

POST /batch  HTTP/1.1
Host: localhost:8080
Content-Type: multipart/mixed; boundary=boundary
--boundary
Content-Type: application/http
Content-ID: &lt;item1&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ &quot;name&quot;: &quot;batch1&quot;, &quot;description&quot;: &quot;batch1 description&quot; }
--boundary
Content-Type: application/http
Content-ID: &lt;item2&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ &quot;name&quot;: &quot;batch2&quot;, &quot;description&quot;: &quot;batch2 description&quot; }
--boundary--

With this it should work, but note that since you are processing parts in the same loop, and calling router.ServeHTTP(w, req) in the end, you're reusing the w writer. What does this mean? If handleItemPost() writes anything to the output, subsequent calls to handleItemPost() can't take that back.

E.g. if a handleItemPost() fails, it responds with an HTTP error (this implies setting response status and writing the body). A subsequent handleItemPost() can't report an error again (headers are already committed), and also if it would report success, the error header is already sent and could only write further message to the error body.

So for example if we modify handleItemPost() to this:

func handleItemPost(w http.ResponseWriter, r *http.Request) {
var item map[string]interface{}
if err := json.NewDecoder(r.Body).Decode(&amp;item); err != nil {
fmt.Printf(&quot;JSON decode error: %v\n&quot;, err)
return
}
fmt.Printf(&quot;Success, item: %v\n&quot;, item)
}

And execute the following curl command:

curl localhost:8080/batch -X POST \
-H &quot;Content-Type: multipart/mixed; boundary=boundary&quot; \
-d &#39;--boundary
Content-Type: application/http
Content-ID: &lt;item1&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ &quot;name&quot;: &quot;batch1&quot;, &quot;description&quot;: &quot;batch1 description&quot; }
--boundary
Content-Type: application/http
Content-ID: &lt;item2&gt;
POST /items HTTP/1.1
Content-Type: application/json
Content-length: 58
{ &quot;name&quot;: &quot;batch2&quot;, &quot;description&quot;: &quot;batch2 description&quot; }
--boundary--&#39;

We will see the following output:

Success, item: map[description:batch1 description name:batch1]
Success, item: map[description:batch2 description name:batch2]

Note that if handleItemPost() needs to remain fully functional and callable on its own (to process the request and produce response), you can't use the same http.ResponseWriter for all of its calls.

In this case you may create and use a separate http.ResponseWriter for each of its invocation. The standard lib has an httptest.ResponseRecorder type that implements http.ResponseWriter. It's primarily for testing purposes, but you may use it here too. It records the written response, so you may inspect it after the call.

For example:

w2 := httptest.NewRecorder()
router.ServeHTTP(w2, req)
if w2.Code != http.StatusOK {
fmt.Printf(&quot;handleItemPost returned non-OK status: %v\n&quot;, w2.Code)
fmt.Printf(&quot;\terror body: %v\n&quot;, w2.Body.String())
}

Running this with your original request (without specifying Content-Length), the output will be:

handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF
handleItemPost returned non-OK status: 400
error body: could not decode item json: EOF

But when you specify the Content-Length of the sub-requests, no output (error) is printed.

huangapple
  • 本文由 发表于 2021年12月20日 18:30:23
  • 转载请务必保留本文链接:https://go.coder-hub.com/70420457.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定