why does golang http ResponseWriter auto add content-length if it's no more than 2kb

huangapple go评论91阅读模式
英文:

why does golang http ResponseWriter auto add content-length if it's no more than 2kb

问题

func (handler Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	var content string
	...
	w.Write([]byte(content))
}

如果 content 的长度小于等于 2048,响应中的 content-length 将会自动添加。如果超过 2048,响应中将没有 content-length,而是会添加 Transfer-Encoding: chunked

我找不到确定 2048 的源代码位置。

我正在寻求帮助找到确定 2048 的源代码位置。

英文:
func (handler Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
	var content string
	...
	w.Write([]byte(content))
}

if len(content) <= 2048, the content-length will be added automatically in the response. And if it's over 2048, there is no content-length, and the Transfer-Encoding: chunked will be added.
I can't find where to determine the 2048.

I'm asking for help to find the source code that where to determine the 2048.

答案1

得分: 5

让我们查看一下http.ResponseWriter接口的文档,以便明确这个特性:
> 如果所有写入数据的总大小小于几KB,并且没有调用Flush方法,Content-Length头部将会自动添加。

首先,我们可以看到这个数字可能不是确切的2048(2 KB),但它在我们预期的“几KB”的范围内。其次,我们可以看到这种行为与Flush方法有关,该方法在Flusher接口中有文档说明:
> Flush将任何缓冲的数据发送到客户端。
>
> Flusher接口由允许HTTP处理程序将缓冲数据刷新到客户端的ResponseWriter实现。
>
> 默认的HTTP/1.x和HTTP/2 ResponseWriter实现支持Flusher,但ResponseWriter包装器可能不支持。处理程序应始终在运行时测试此功能。

正如文档所说,你的ResponseWriter可能支持数据缓冲和刷新。这意味着当你向响应写入数据时,它不会立即通过连接传输。相反,它首先被写入到缓冲区中。每当缓冲区已满无法继续写入,并且当ServeHTTP方法返回时,整个缓冲区将被传输。这确保了即使进行大量的小写操作,数据也能高效地传输,并且所有数据最终都会被传输。你还可以使用Flush方法随时主动清空缓冲区。HTTP头部必须在正文数据之前发送,但在第一次清空缓冲区之前无需发送它们。

将所有这些内容结合起来,你会发现如果总写入量不超过缓冲区大小,并且我们从未调用Flush,那么在所有数据准备好之前,头部不需要发送,此时我们才知道内容长度。如果总写入量超过缓冲区大小,则在知道内容长度之前必须发送头部,因此ResponseWriter无法自动确定它。

这在net/http/server.go中实现。具体来说,这里是缓冲区大小的声明以及实现部分缓冲写入行为的chunkedWriter

// This should be &gt;= 512 bytes for DetectContentType,
// but otherwise it&#39;s somewhat arbitrary.
const bufferBeforeChunkingSize = 2048

// chunkWriter writes to a response&#39;s conn buffer, and is the writer
// wrapped by the response.w buffered writer.
//
// chunkWriter also is responsible for finalizing the Header, including
// conditionally setting the Content-Type and setting a Content-Length
// in cases where the handler&#39;s final output is smaller than the buffer
// size. It also conditionally adds chunk headers, when in chunking mode.
//
// See the comment above (*response).Write for the entire write flow.
type chunkWriter struct {

链接到源代码,适用于1.19.5版本。请注意,源代码可能会随着每个Go版本的更改而变化。

英文:

Let's look at the documentation of this feature in the http.ResponseWriter interface just for clarity:
> [I]f the total size of all written data is under a few KB and there are no Flush calls, the Content-Length header is added automatically.

First, we can see that the number may not be exactly 2048 (2 KB), but that is in the range we would expect for "a few KB". Second, we can see that this behaviour has something to do with the Flush method, which is documented in the Flusher interface:
> Flush sends any buffered data to the client.
>
> The Flusher interface is implemented by ResponseWriters that allow an HTTP handler to flush buffered data to the client.
>
> The default HTTP/1.x and HTTP/2 ResponseWriter implementations support Flusher, but ResponseWriter wrappers may not. Handlers should always test for this ability at runtime.

As it says, your ResponseWriter may support data buffering and flushing. What this means is that when you write data to the response writer, it does not immediately get transmitted over the connection. Instead, it first gets written into a buffer. Each time the buffer is too full to write to anymore, and when the ServeHTTP method returns, the whole buffer will get transmitted. This ensures that data gets transmitted efficiently even when you do lots of tiny writes, and that all data gets transmitted in the end. You also have the option of proactively emptying the buffer at any time with the Flush method. The HTTP headers must be sent before the body data, but there's no need to send them until the first time the buffer is emptied.

Putting all this together, you'll see that if the total amount written is no more than the buffer size, and we never call Flush, then the headers do not need to be sent until all the data is ready, at which point we know the content length. If the total amount written is more than the buffer size, then the headers must be sent before the content length is known, and so the ResponseWriter can't determine it automatically.

This is implemented in the source in net/http/server.go. Specifically, here are the declarations of the buffer size, and the chunkedWriter which implements part of the buffered writing behaviour:

// This should be &gt;= 512 bytes for DetectContentType,
// but otherwise it&#39;s somewhat arbitrary.
const bufferBeforeChunkingSize = 2048

// chunkWriter writes to a response&#39;s conn buffer, and is the writer
// wrapped by the response.w buffered writer.
//
// chunkWriter also is responsible for finalizing the Header, including
// conditionally setting the Content-Type and setting a Content-Length
// in cases where the handler&#39;s final output is smaller than the buffer
// size. It also conditionally adds chunk headers, when in chunking mode.
//
// See the comment above (*response).Write for the entire write flow.
type chunkWriter struct {

Link to the source code for 1.19.5. Please note that the source code is subject to change with each Go release.

答案2

得分: 3

该值在此处定义(https://github.com/golang/go/blob/245e95dfabd77f337373bf2d6bb47cd353ad8d74/src/net/http/server.go#L338):

// 这个值应该是 >= 512 字节用于 DetectContentType,
// 但除此之外它有些随意。
const bufferBeforeChunkingSize = 2048

Write的生命周期解释了发生的情况:

如果处理程序没有事先声明 Content-Length,我们要么进入分块模式,要么如果处理程序在分块缓冲区大小之前完成运行,则计算一个 Content-Length 并将其发送到标头中。

英文:

The value is defined here:

// This should be &gt;= 512 bytes for DetectContentType,
// but otherwise it&#39;s somewhat arbitrary.
const bufferBeforeChunkingSize = 2048

The Life of a Write explains what happens:

> If the handler didn't declare a Content-Length up front, we either go into chunking mode or, if the handler finishes running before the chunking buffer size, we compute a Content-Length and send that in the header instead.

huangapple
  • 本文由 发表于 2023年1月12日 11:51:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/75091383.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定