Go语言的http标准库中是否存在内存泄漏问题?

huangapple go评论95阅读模式
英文:

Memory leak in Go http standard library?

问题

请稍等,我会为你翻译这段代码和问题。

英文:

Have a Go binary implement an http server:

package main

import (
    "net/http"
)

func main() {
    http.ListenAndServe(":8080", nil)
}

It will start with ~850 kb or so of memory. Send it some requests via your web browser. Observe it quickly rises to 1 mb. If you wait, you'll see it never goes down. Now hammer it with Apache Bench (using the script below) and see your memory usage continually increase. After sometime it will eventually plateau at around 8.2 MB or so.

Edit: It doesn't seem to stop at 8.2, rather it slows down significantly. It's currently at 9.2 and still rising.

In short, why is this happening? I used this shell script:

while [ true ]
do
    ab -n 1000 -c 100 http://127.0.0.1:8080/
    sleep 1
end

While trying to get to the bottom of this, I've tried to tweak the settings. I've tried closing using r.Close = true to prevent Keep-Alive. Nothing seems to work.

I found this originally while trying to determine if there was a memory leak in a program I'm writing. It has a lot of http handlers and I/O calls. After checking I had closed all my database connections I kept seeing it's memory usage rise. My program to plateau at around 433 MB.

Here's the output of Goenv:

GOARCH="amd64"
GOBIN=""
GOCHAR="6"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mark/Documents/Programming/Go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
TERM="dumb"
CC="clang"
GOGCCFLAGS="-g -O2 -fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fno-common"
CXX="clang++"
CGO_ENABLED="1"

答案1

得分: 19

根据你在评论中提供的堆pprof,看起来你通过gorilla/sessionsgorilla/context泄漏了内存(接近400MB)。

请参考这个邮件列表线程:https://groups.google.com/forum/#!msg/gorilla-web/clJfCzenuWY/N_Xj9-5Lk6wJ 和这个GitHub问题:https://github.com/gorilla/sessions/issues/15

下面是一个泄漏非常快的版本:

package main

import (
	"fmt"
	// "github.com/gorilla/context"
	"github.com/gorilla/sessions"
	"net/http"
)

var (
	cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
	http.HandleFunc("/", defaultHandler)
	http.ListenAndServe(":8080", nil)
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
	cookieStore.Get(r, "leak-test")
	fmt.Fprint(w, "Hi there")
}

下面是一个清理并且具有相对静态RSS的版本:

package main

import (
	"fmt"
	"github.com/gorilla/context"
	"github.com/gorilla/sessions"
	"net/http"
)

var (
	cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
	http.HandleFunc("/", defaultHandler)
	http.ListenAndServe(":8080", context.ClearHandler(http.DefaultServeMux))
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
	cookieStore.Get(r, "leak-test")
	fmt.Fprint(w, "Hi there")
}

希望对你有帮助!

英文:

From the heap pprof you have provided in comments, it looks like you are leaking memory via gorilla/sessions and gorilla/context (almost 400MB).

Refer to this ML thread: https://groups.google.com/forum/#!msg/gorilla-web/clJfCzenuWY/N_Xj9-5Lk6wJ and this GH issue: https://github.com/gorilla/sessions/issues/15

Here is a version that leaks extremely quickly:

package main

import (
	"fmt"
	// "github.com/gorilla/context"
	"github.com/gorilla/sessions"
	"net/http"
)

var (
	cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
	http.HandleFunc("/", defaultHandler)
	http.ListenAndServe(":8080", nil)
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
	cookieStore.Get(r, "leak-test")
	fmt.Fprint(w, "Hi there")
}

Here is one that cleans up and has a relatively static RSS:

package main

import (
	"fmt"
	"github.com/gorilla/context"
	"github.com/gorilla/sessions"
	"net/http"
)

var (
	cookieStore = sessions.NewCookieStore([]byte("cookie-secret"))
)

func main() {
	http.HandleFunc("/", defaultHandler)
	http.ListenAndServe(":8080", context.ClearHandler(http.DefaultServeMux))
}

func defaultHandler(w http.ResponseWriter, r *http.Request) {
	cookieStore.Get(r, "leak-test")
	fmt.Fprint(w, "Hi there")
}

huangapple
  • 本文由 发表于 2014年1月13日 05:41:55
  • 转载请务必保留本文链接:https://go.coder-hub.com/21080642.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定