这段代码是否可以避免大型的HTTP请求? Golang

huangapple go评论74阅读模式
英文:

Is this code ok to avoid a big HTTP request? Golang

问题

我目前正在学习使用Go语言作为服务器端语言。我正在学习如何处理表单,所以我想知道如何防止恶意客户端发送非常大的文件(在multipart/form-data表单的情况下),导致服务器内存耗尽。目前这是我在stackoverflow上找到的代码:

part, _ := ioutil.ReadAll(io.LimitReader(r.Body, 8388608))

r.Body = ioutil.NopCloser(io.MultiReader(bytes.NewReader(part), r.Body))

在我的代码中,r等于*http.Request。所以,我认为这段代码运行良好,但问题是,无论我发送多大的文件(根据我的代码,最大大小为8M),我的代码仍然接收整个文件,所以我对我的代码是否正常工作存在疑问。所以我的问题是,我的代码真的有问题吗?我是否缺少某个概念,导致我认为我的代码出现故障?如何正确限制HTTP请求的大小?

更新

我尝试运行答案中显示的代码,也就是这段代码:

part, _ := ioutil.ReadAll(io.LimitReader(r.Body, 8388608))

r.Body = ioutil.NopCloser(bytes.NewReader(part))

但是当我运行该代码,并发送一个大于8M的文件时,我的Web浏览器显示以下消息:

连接被重置

在页面加载时,与服务器的连接被重置。

我该如何解决这个问题?如何最大限度地读取8M,但又不出现该错误?

英文:

I am currently learning to use golang as a server side language. I'm learning how to handle forms, and so I wanted to see how I could prevent some malicious client from sending a very large (in the case of a form with multipart/form-data) file and causing the server to run out of memory. For now this is my code which I found in a question here on stackoverflow:

part, _ := ioutil.ReadAll(io.LimitReader(r.Body, 8388608))

r.Body = ioutil.NopCloser(io.MultiReader(bytes.NewReader(part), r.Body))

In my code r is equal to *http.Request. So, I think that code works well, but what happens is that when I send a file regardless of its size (according to my code, the maximum size is 8M) my code still receives the entire file, so I have doubts that my code actually works. So my question is. Does my code really work wrong? Is there a concept that I am missing and that is why I think my code is malfunctioning? How can I limit the size of an http request correctly?

Update

I tried to run the code that was shown in the answers, I mean, this code:

part, _ := ioutil.ReadAll(io.LimitReader(r.Body, 8388608))

r.Body = ioutil.NopCloser(bytes.NewReader(part))

But when I run that code, and when I send a file larger than 8M I get this message from my web browser:

> The connection was reset
>
> The connection to the server was reset while the page was loading.

How can I solve that? How can I read only 8M maximum but without getting that error?

答案1

得分: 2

我会翻译以下内容:

我想问一个问题:“如果您的服务接收到超过最大大小的请求,它的行为预期是怎样的?”

也许您可以简单地检查请求的ContentLength,如果超过了最大值,立即返回400 Bad Request

func MyHandler(rw http.ResponseWriter, rq *http.Request) {
   if rq.ContentLength > 8388608 {
      rw.WriteHeader(http.StatusBadRequest)
      rw.Write([]byte("请求内容超过限制"))
      return
   }

   // ... 正常处理
}

这样做的好处是,在最早的可能机会上(除了入口处的一些限流),不需要读取任何内容并决定不继续进行,从而最大限度地减少了进程的 CPU 和内存负载。

它还简化了正常处理过程,因为不需要考虑可能涉及部分请求的情况,或者在请求内容限制达到之前终止并可能需要清理处理过程的情况。

英文:

I would ask the question: "How is your service intended/expected to behave if it receives a request greater than the maximum size?"

Perhaps you could simply check the ContentLength of the request and immediately return a 400 Bad Request if it exceeds your maximum?

func MyHandler(rw http.ResponseWriter, rq *http.Request) {
   if rq.ContentLength > 8388608 {
      rw.WriteHeader(http.StatusBadRequest)
      rw.Write([]byte("request content limit exceeded"))
      return
   }

   // ... normal processing
}

This has the advantage of not reading anything and deciding not to proceed at the earliest possible opportunity (short of some throttling on the ingress itself), minimising cpu and memory load on your process.

It also simplifies your normal processing which then does not have to be concerned with catering for circumstances where a partial request might be involved, or aborting and possibly having to clean up processing if the request content limit is reached before all content has been processed..

答案2

得分: 1

你的代码中有一段:

r.Body = ioutil.NopCloser(io.MultiReader(bytes.NewReader(part), r.Body))

这意味着你给r.Body赋了一个新的io.MultiReader,它会:

  1. 从内存中的字节切片中最多读取8388608个字节
  2. 然后继续读取剩余的字节

为了确保你只读取最多8388608个字节,你可以将那行代码替换为:

r.Body = ioutil.NopCloser(bytes.NewReader(part))
英文:

Your code reads:

r.Body = ioutil.NopCloser(io.MultiReader(bytes.NewReader(part), r.Body))

This means that you are assigned a new io.MultiReader to your body that:

  1. reads at most 8388608 from a byte slice in memory
  2. and then reads the rest of the body after those 8388608 bytes

To ensure that you only read 8388608 bytes at most, replace that line with:

r.Body = ioutil.NopCloser(bytes.NewReader(part))

huangapple
  • 本文由 发表于 2022年11月18日 10:15:42
  • 转载请务必保留本文链接:https://go.coder-hub.com/74484230.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定