英文:
Does delete() instantly frees memory or runtime.GC() is required to free it up?
问题
我有一个地图。
myMap := map[string]string
myMap['hello'] = 'world'
myMap['foo'] = 'bar'
当我从myMap中删除一个元素时,比如:
delete(myMap['hello'])
它是立即释放内存还是在垃圾回收器运行后释放内存?如果在垃圾回收器运行后释放内存,运行runtime.GC()
会立即清理内存吗?
另外,runtime.GC()
会消耗很多资源吗?在每次delete()
函数之后运行runtime.GC()
是否可行?
更新2:
忘记我的程序做了什么(基本上是更新1)
请查看此链接http://play.golang.org/p/Wb8-4qWyf4
有一个子程序每10微秒向地图添加一次
有一个子程序每1微秒从地图中删除一次(比添加快10倍)
在本地机器上运行此程序,您会发现它会慢慢占用更多的RAM。(慢慢是因为我添加了一个睡眠时间,否则计算机会卡住)
更新1:
我的程序每分钟从数据库中获取5000行数据,并将其存储在名为datastore的地图中。
有100个子程序正在处理datastore中的每一行数据。处理一行数据需要很长时间(不到一秒)。
如果数据成功处理,则从datastore中删除它,但在下一分钟内会获取下一个5000行并添加到datastore中。
我在datastore中保留最多20000行数据,这不多(最多200 MB)。
处理数百万行数据后,应用程序开始占用100%的RAM,并在最后被内核终止。
如果delete()能立即清除内存,这种情况不应该发生。
英文:
I have a map
myMap := map[string]string
myMap['hello'] = 'world'
myMap['foo'] = 'bar'
When I delete an element from myMap like,
delete(myMap['hello'])
Does it instantly frees up memory or does it frees memory after garbage collector is run. <br>
If it frees up memory after garbage collector is run, Does running runtime.GC()
will clean up the memory instantly.
Also is runtime.GC()
resource hungry ? or Its okay to run runtime.GC()
after every delete()
function
Update 2:<br>
Forget what my program does (basically update 1) <br>
Check this link http://play.golang.org/p/Wb8-4qWyf4 <br>
There is a subroutine to add to a Map every 10 Microsecond <br>
There is a subroutine to delete from map every 1 Microsecond (10 times faster that adding) <br>
Run this program in your local machine, you will find that its keeps on occupying more and <br>more RAM slowly. (Slowly because I added a sleep time, otherwise computer will hang)
<br>
Update 1
<br>
My program fetches 5000 rows of data from database every minute and stores it in a map called datastore.<br>
There are 100 subroutines running which processes each rows from datastore. It takes much time to process one row (less than a second)<br>
If data is successfully processed it is deleted "delete()" from datastore, but within next minute next 5000 is fetched and added to datastore.<br>
I am keeping maximum of 20,000 rows in datastore. Which is not much (200 MB at max)<br>
After processing millions of rows, application start taking 100% of RAM and gets Killed by kernal at the end.
It should not happen if delete() was clearing the memory instantly.
答案1
得分: 1
你的deleter
只执行了一次。此外,map
的访问不是并发安全的,所以你需要在代码周围添加互斥锁。
这里是一个修复方法,它不会随着时间的推移而增长:http://play.golang.org/p/GWQ2hJiySP
package main
import (
"fmt"
"sync"
"time"
)
var datastore = make(map[int64]string)
var m sync.Mutex
func adder() {
var count int64 = 0
for {
m.Lock()
datastore[count] = "kjnbhjsdhgvsaghbsdbasjsabjhsabasbdjashdbashdbjasbdhasbdjbdjbdjhabjds"
m.Unlock()
count++
time.Sleep(10 * time.Microsecond)
}
}
func deleter() {
for {
m.Lock()
for key, _ := range datastore {
delete(datastore, key)
time.Sleep(1 * time.Microsecond)
}
m.Unlock()
}
}
func main() {
// Start adding stuff to datastore (with MORE sleep time = 10 Microsecond)
go adder()
// Wait for some time
time.Sleep(1 * time.Second)
// Start deleting stuff from datastore (With LESS sleep time = 1 Microsecond)
go deleter()
time.Sleep(1 * time.Hour)
fmt.Println("Done")
}
我在deleter
中将锁放在了整个循环周围。实际上,只在delete
周围放置锁就足够了,但是你不应该使用range
来删除条目,因为修改map可能会改变迭代的顺序,所以这种结构不应该在实际程序中使用。
英文:
You are executing your deleter
only once. In addition, map
access is not concurrent-safe so you need to add Mutex around your code.
Here is a fix, that will not grow with time: http://play.golang.org/p/GWQ2hJiySP
package main
import (
"fmt"
"sync"
"time"
)
var datastore = make(map[int64]string)
var m sync.Mutex
func adder() {
var count int64 = 0
for {
m.Lock()
datastore[count] = "kjnbhjsdhgvsaghbsdbasjsabjhsabasbdjashdbashdbjasbdhasbdjbdjbdjhabjds"
m.Unlock()
count++
time.Sleep(10 * time.Microsecond)
}
}
func deleter() {
for {
m.Lock()
for key, _ := range datastore {
delete(datastore, key)
time.Sleep(1 * time.Microsecond)
}
m.Unlock()
}
}
func main() {
// Start adding stuff to datastore (with MORE sleep time = 10 Microsecond)
go adder()
// Wait for some time
time.Sleep(1 * time.Second)
// Start deleting stuff from datastore (With LESS sleep time = 1 Microsecond)
go deleter()
time.Sleep(1 * time.Hour)
fmt.Println("Done")
}
I have put the lock in the deleter
around the whole loop. It should be only necessary around the delete
, however, you should not delete entries that way, with a range, because modifying the map can change the order of the iteration, so this structure should not be used in a real-life program.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论