How can I get a size of request.Header in bytes in Golang?

huangapple go评论87阅读模式
英文:

How can I get a size of request.Header in bytes in Golang?

问题

我需要找到request.Header的大小,其中request的类型为*http.Request

req, err := http.NewRequest("GET", "/", nil)
cookie := &http.Cookie{Name: "foo", Value: "bar"}
req.AddCookie(cookie)

我尝试了以下方法:

len(request.Header) // 返回map中的元素数量,也就是header的数量

for k, v := range req.Header {
  bytesSize += len(k) + len(v)
}

但是这些方法都不起作用,因为v是一个map。

我在https://stackoverflow.com/questions/31847549/computing-the-memory-footprint-or-byte-length-of-a-map 上找到了一个问题,但是答案似乎相当复杂(而且他们的map值是整数,这里不是这种情况)。

更新:实际上,这里的Header的定义是type Header map[string][]string,所以我们不需要使用递归。

英文:

I need to find the size of request.Header where request has *http.Request type:

req, err := http.NewRequest("GET", "/", nil)
cookie := &http.Cookie{Name: "foo", Value: "bar"}
req.AddCookie(cookie)

I tried

len(request.Header) # returned the number of elements in the map -- essentially the number of headers

and

for k, v := range req.Header {
  bytesSize += len(k) + len(v)
}

that didn't work either since v was a map.

I found https://stackoverflow.com/questions/31847549/computing-the-memory-footprint-or-byte-length-of-a-map question but the answer seems pretty complicated (and their map values are integers which is not the case here).

Update: actually here's the definition of type Header map[string][]string so we don't have to use recursion.

答案1

得分: 3

https://pkg.go.dev/net/http#Server.MaxHeaderBytes 可以帮助你处理这个问题。

这个示例在 Playground 中无法可靠地工作(拨号或连接超时)。但在本地似乎可以可靠地工作,这让我猜测这是 Playground 行为的一个副作用。

我们将启动一个具有较大 MaxHeaderBytes 的 http 服务器,然后超过它。

在这里,我将 MaxHeaderBytes 设置为一个相当小的值。我在我的 X-Long-Header: long long long .... header 中传递了远远超过该值的内容。如果你可以让 Playground 工作(只需运行几次)或在本地运行它,你将得到:

&{431 Request Header Fields Too Large 431 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8]] 0xc00001a180 -1 [] true false map[] 0xc000176000 <nil>}
Body: 431 Request Header Fields Too Large

如你所见,如果所有的头部都太大,就会自动生成 431。

如果特定的头部太长,你的处理程序本身可能会适当地响应 431,但是当你的处理程序接收到一个 http.Request 时,头部已经被接收到了。尝试计算头部的总长度,然后根据此长度响应 431 是没有意义的。

此外,标准头部可能会出现变化,因此限制整体头部大小过于严格是不明智的。

相反,检查你关心的每个单独的头部。

英文:

https://pkg.go.dev/net/http#Server.MaxHeaderBytes can handle this for you.

This demo doesn't work reliably in the Playground (dial or connect timeouts) . It seems to work reliably locally though, which makes me guess it's an artifact of the playground's behavior.

We'll start an http Server with alow MaxHeaderBytes and then surpass it greatly.

package main

import (
	"context"
	"fmt"
	"io"
	"net"
	"net/http"
	"strings"
	"time"
)

func main() {
	res := make(chan error)
	mux := http.NewServeMux()
	mux.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
		fmt.Fprintf(w, "%+v", r.Header)
	})
	s := &http.Server{
		Addr:           "127.0.0.1:8103",
		Handler:        mux,
		ReadTimeout:    1 * time.Second,
		WriteTimeout:   1 * time.Second,
		MaxHeaderBytes: 2048,
	}
	if l, err := net.Listen("tcp", "127.0.0.1:8103"); err != nil {
		panic(fmt.Errorf("Couldn't listen: %w", err))
	} else {
		go func() {
			res <- s.Serve(l)
		}()
	}
	client := &http.Client{
		Timeout: 3 * time.Second,
	}
	req, err := http.NewRequest("GET", "http://127.0.0.1:8103", nil)
	if err != nil {
		panic(err)
	}
	req.Header.Add("X-Long-Header", strings.Repeat("long ", 2048)+"header")
	resp, err := client.Do(req)
	if err != nil {
		panic(fmt.Errorf("HTTP Request failed: %w", err))
	}
	fmt.Println(resp)
	body, err := io.ReadAll(resp.Body)
	if err != nil {
		panic(fmt.Errorf("Could not read response body: %w", err))
	}
	fmt.Println("Body:", string(body))
	s.Shutdown(context.Background())
	<-res
}

Here, I'm setting MaxHeaderBytes to a fairly small value. I am passing far more than that value in my X-Long-Header: long long long .... header. If you can get the playground to work (just run it a few times) or run it locally, you'll get:

&{431 Request Header Fields Too Large 431 HTTP/1.1 1 1 map[Content-Type:[text/plain; charset=utf-8]] 0xc00001a180 -1 [] true false map[] 0xc000176000 <nil>}
Body: 431 Request Header Fields Too Large

As you can see, the 431 will automatically be generated if all headers are too large.

It might be appropriate for your handler itself to respond with a 431 if particular headers were too long, but by the time your handler has been passed an http.Request, the headers have been received. It doesn't make sense to try to compute the total length of the headers yourself and then respond with a 431 based on that.

Besides, standard headers may come and go, so it would be unwise to restrict overall header size too closely.

Instead, check whatever individual headers you're concerned about.

huangapple
  • 本文由 发表于 2021年10月27日 06:59:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/69730864.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定