英文:
is ioutil.ReadAll blocking my server?
问题
我正在尝试使用net/http包在Go中编写一个服务器。我只有一个路由，而且非常简单。它从S3下载文件并将其返回给客户端：
response, err := http.Get("some S3 url")
if err != nil {
return
}
body, err := ioutil.ReadAll(response.Body)
w.Write(body)
自己下载该URL大约需要0.25秒。所以我启动这个服务器并以每秒250个请求的速度发送请求。最初，我在0.25秒内收到响应。但是这个数字不断增加，直到响应需要45秒。我在一个40核心的机器上运行这个程序，使用GOMAXPROCS=40。我开始怀疑下载是否没有并行进行。
但是，如果我注释掉这一行：
body, err := ioutil.ReadAll(response.Body)
并且只返回一些相同长度的垃圾数据，突然间我的服务器始终在0.25秒内响应。为什么删除ReadAll后速度更快？
英文:
I'm trying to write a server in Go, using the net/http package. I only have one route, and it's pretty simple. It downloads a file from S3 and returns it to the client:
response, err := http.Get("some S3 url")
if err != nil {
return
}
body, err := ioutil.ReadAll(response.Body)
w.Write(body)
Downloading the url myself takes about 0.25 seconds. So I start this server and send it 250 requests/sec. Initially I get responses back within 0.25 seconds. But that number keeps going up until it starts taking 45 seconds for a response. I'm running this on a 40 core machine, with GOMAXPROCS=40. I started wondering if somehow the downloads aren't happening in parallel.
But if I comment out this line:
body, err := ioutil.ReadAll(response.Body)
And just return some garbage data of equal length, suddenly my server consistently responds in 0.25 seconds. Why is it faster after removing the ReadAll?
答案1
得分: 2
几个问题需要考虑：
- 你没有关闭
response.Body，导致服务器的文件描述符用尽。 - 垃圾回收器运行缓慢，使用
ReadAll读取太多文件导致内存不足。 - 由于问题 #1，网络传输受阻。
尝试使用以下代码，看看是否有所帮助：
response, err := http.Get("some S3 url")
if err != nil {
return
}
defer response.Body.Close()
_, err := io.Copy(w, response.Body)
英文:
Few things comes to mind:
- You're not closing
response.Bodyand the server is running out of FDs. - The garbage collector is being slow and you're running out of memory for reading so many files with
ReadAll. - You're choking the networking because of #1.
Try something like this and see if it helps:
response, err := http.Get("some S3 url")
if err != nil {
return
}
defer response.Body.Close()
_, err := io.Copy(w, response.Body)
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论