英文:
Memory leak in Go http standard library?
问题
请稍等,我会为你翻译这段代码和问题。
英文:
Have a Go binary implement an http server:
package main
import (
"net/http"
)
func main() {
http.ListenAndServe(":8080", nil)
}
It will start with ~850 kb or so of memory. Send it some requests via your web browser. Observe it quickly rises to 1 mb. If you wait, you'll see it never goes down. Now hammer it with Apache Bench (using the script below) and see your memory usage continually increase. After sometime it will eventually plateau at around 8.2 MB or so.
Edit: It doesn't seem to stop at 8.2, rather it slows down significantly. It's currently at 9.2 and still rising.
In short, why is this happening? I used this shell script:
while [ true ]
do
ab -n 1000 -c 100 http://127.0.0.1:8080/
sleep 1
end
While trying to get to the bottom of this, I've tried to tweak the settings. I've tried closing using r.Close = true
to prevent Keep-Alive. Nothing seems to work.
I found this originally while trying to determine if there was a memory leak in a program I'm writing. It has a lot of http handlers and I/O calls. After checking I had closed all my database connections I kept seeing it's memory usage rise. My program to plateau at around 433 MB.
Here's the output of Goenv:
GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mark/Documents/Programming/Go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
TERM="dumb"
CC="clang"
GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fno-common"
CXX="clang++"
CGO_ENABLED="1"
答案1
得分: 19
根据你在评论中提供的堆pprof
,看起来你通过gorilla/sessions
和gorilla/context
泄漏了内存(接近400MB)。
请参考这个邮件列表线程:https://groups.google.com/forum/#!msg/gorilla-web/clJfCzenuWY/N_Xj9-5Lk6wJ 和这个GitHub问题:https://github.com/gorilla/sessions/issues/15
下面是一个泄漏非常快的版本:
package main
import (
"fmt"
// "github.com/gorilla/context"
"github.com/gorilla/sessions"
"net/http"
)
var (
cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)
func main() {
http.HandleFunc("/", defaultHandler)
http.ListenAndServe(":8080", nil)
}
func defaultHandler(w http.ResponseWriter, r *http.Request) {
cookieStore.Get(r, "leak-test")
fmt.Fprint(w, "Hi there")
}
下面是一个清理并且具有相对静态RSS的版本:
package main
import (
"fmt"
"github.com/gorilla/context"
"github.com/gorilla/sessions"
"net/http"
)
var (
cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)
func main() {
http.HandleFunc("/", defaultHandler)
http.ListenAndServe(":8080", context.ClearHandler(http.DefaultServeMux))
}
func defaultHandler(w http.ResponseWriter, r *http.Request) {
cookieStore.Get(r, "leak-test")
fmt.Fprint(w, "Hi there")
}
希望对你有帮助!
英文:
From the heap pprof
you have provided in comments, it looks like you are leaking memory via gorilla/sessions
and gorilla/context
(almost 400MB).
Refer to this ML thread: https://groups.google.com/forum/#!msg/gorilla-web/clJfCzenuWY/N_Xj9-5Lk6wJ and this GH issue: https://github.com/gorilla/sessions/issues/15
Here is a version that leaks extremely quickly:
package main
import (
"fmt"
// "github.com/gorilla/context"
"github.com/gorilla/sessions"
"net/http"
)
var (
cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)
func main() {
http.HandleFunc("/", defaultHandler)
http.ListenAndServe(":8080", nil)
}
func defaultHandler(w http.ResponseWriter, r *http.Request) {
cookieStore.Get(r, "leak-test")
fmt.Fprint(w, "Hi there")
}
Here is one that cleans up and has a relatively static RSS:
package main
import (
"fmt"
"github.com/gorilla/context"
"github.com/gorilla/sessions"
"net/http"
)
var (
cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)
func main() {
http.HandleFunc("/", defaultHandler)
http.ListenAndServe(":8080", context.ClearHandler(http.DefaultServeMux))
}
func defaultHandler(w http.ResponseWriter, r *http.Request) {
cookieStore.Get(r, "leak-test")
fmt.Fprint(w, "Hi there")
}
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论