英文:
channels and memory leaks
问题
我正在尝试开发一个连续运行的程序。
它应该每隔sleepPool秒从数据库中获取一些数据，并以非阻塞的方式处理信息（至少我是这样尝试的）。
问题是内存一直在增长，所以我想知道我是否做错了什么。
以下是我的程序的一部分代码。
var uCh = make(chan *user, buffLimit) // 发送要处理的新用户
var statsCh = make(chan *user, buffLimit) // 发送要存储的新用户
func main() {
go emitUser(db)
go consumeUser(db)
for ur := range statsCh {
log.Infoln(ur)
}
}
func emitUser(db *sql.DB) {
for {
time.Sleep(sleepPool * time.Second)
log.Infoln("查找新用户")
rows, err := rowsD.Query()
for rows.Next() {
uCh <- usr
}
}
}
func consumeUser(db *sql.DB) {
for usr := range uCh {
go func(usr *user) {
// 处理用户
statsCh <- usr
}(usr)
}
}
我读到可能需要关闭通道，以便垃圾回收器可以回收内存，但我不确定如何做到这一点（因为程序应该连续运行），而且我真的需要这样做吗？因为数据总是被读取（由main中的范围保证），所以我认为内存会被回收。
英文:
I'm trying to develop a program which runs continuously.
It should pull some data from a database every sleepPool seconds and 'process' the information in a non-blocking way(at least that's what I'm trying to do).
The issue is that the memory keeps growing so I'm wondering if I'm doing something wrong.
Below is a snippet from the my program.
var uCh = make(chan *user, buffLimit) //emits new users to process
var statsCh = make(chan *user, buffLimit) //emits new users to store
func main() {
go emitUser(db)
go consumeUser(db)
for ur := range statsCh {
log.Infoln(ur)
}
}
func emitUser(db *sql.DB) {
for {
time.Sleep(sleepPool * time.Second)
log.Infoln("looking for new users")
rows, err := rowsD.Query()
for rows.Next() {
uCh <- usr
}
}
}
func consumeUser(db *sql.DB) {
for usr := range uCh {
go func(usr *user) {
//do something with the user
statsCh <- usr
}(usr)
}
}
I've read that I may need to close the channels so that the gc can recycle the memory but I'm not sure how to do that (because the program should run continuously) and if I really need to do it because the data is always read (guaranteed by the range from main) so I assume the memory is recycled.
答案1
得分: 3
你没有给垃圾回收足够的时间来启动，等待一个小时后再检查内存。
如果你真的非常想（不好的主意，会减慢程序速度）强制释放内存，你可以使用类似以下的代码：
import "runtime/debug"
//........
func forceFree() {
for _ = range time.Tick(30 * time.Second) {
debug.FreeOSMemory()
}
}
func init() {
go forceFree()
}
英文:
You didn't give enough time for the GC to kick in, wait for an hour then check the memory.
If you really really wan (bad idea and gonna slow your program) to force it to free the memory you can use something like:
import "runtime/debug"
//........
func forceFree() {
for _ = range time.Tick(30 * time.Second) {
debug.FreeOSMemory()
}
}
func init() {
go forceFree()
}
答案2
得分: 0
这段代码只是在进行一些无害的传递操作，但不应该泄漏任何信息。
我猜测rowsD.Query方法可能有一些缓存或其他泄漏问题。
当然，也有可能是内存碎片化或垃圾回收器的副作用（在这种情况下，你应该看到内存使用量随着时间的推移稳定下来，甚至下降）。
英文:
This code does a lot of harmless passing things around, but should not leak anything.
My guess is that the rowsD.Query method has some kind of cache, or other leak.
Of course, it could just be fragmentation, or an artefact of the garbage collector (in which case you should see memory use level off, or even drop, over time).
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论