delete()函数是否立即释放内存,还是需要使用runtime.GC()来释放内存?

huangapple go评论97阅读模式
英文:

Does delete() instantly frees memory or runtime.GC() is required to free it up?

问题

我有一个地图。

myMap := map[string]string
myMap['hello'] = 'world'
myMap['foo'] = 'bar'

当我从myMap中删除一个元素时,比如:

delete(myMap['hello'])

它是立即释放内存还是在垃圾回收器运行后释放内存?如果在垃圾回收器运行后释放内存,运行runtime.GC()会立即清理内存吗?

另外,runtime.GC()会消耗很多资源吗?在每次delete()函数之后运行runtime.GC()是否可行?

更新2:
忘记我的程序做了什么(基本上是更新1)
请查看此链接http://play.golang.org/p/Wb8-4qWyf4
有一个子程序每10微秒向地图添加一次
有一个子程序每1微秒从地图中删除一次(比添加快10倍)
在本地机器上运行此程序,您会发现它会慢慢占用更多的RAM。(慢慢是因为我添加了一个睡眠时间,否则计算机会卡住)

更新1:
我的程序每分钟从数据库中获取5000行数据,并将其存储在名为datastore的地图中。
有100个子程序正在处理datastore中的每一行数据。处理一行数据需要很长时间(不到一秒)。
如果数据成功处理,则从datastore中删除它,但在下一分钟内会获取下一个5000行并添加到datastore中。
我在datastore中保留最多20000行数据,这不多(最多200 MB)。
处理数百万行数据后,应用程序开始占用100%的RAM,并在最后被内核终止。
如果delete()能立即清除内存,这种情况不应该发生。

英文:

I have a map

myMap := map[string]string
myMap['hello'] = 'world'
myMap['foo'] = 'bar'

When I delete an element from myMap like,

delete(myMap['hello'])

Does it instantly frees up memory or does it frees memory after garbage collector is run. <br>
If it frees up memory after garbage collector is run, Does running runtime.GC() will clean up the memory instantly.

Also is runtime.GC() resource hungry ? or Its okay to run runtime.GC() after every delete() function

Update 2:<br>
Forget what my program does (basically update 1) <br>
Check this link http://play.golang.org/p/Wb8-4qWyf4 <br>
There is a subroutine to add to a Map every 10 Microsecond <br>
There is a subroutine to delete from map every 1 Microsecond (10 times faster that adding) <br>
Run this program in your local machine, you will find that its keeps on occupying more and <br>more RAM slowly. (Slowly because I added a sleep time, otherwise computer will hang)
<br>

Update 1
<br>
My program fetches 5000 rows of data from database every minute and stores it in a map called datastore.<br>
There are 100 subroutines running which processes each rows from datastore. It takes much time to process one row (less than a second)<br>
If data is successfully processed it is deleted "delete()" from datastore, but within next minute next 5000 is fetched and added to datastore.<br>
I am keeping maximum of 20,000 rows in datastore. Which is not much (200 MB at max)<br>
After processing millions of rows, application start taking 100% of RAM and gets Killed by kernal at the end.
It should not happen if delete() was clearing the memory instantly.

答案1

得分: 1

你的deleter只执行了一次。此外,map的访问不是并发安全的,所以你需要在代码周围添加互斥锁。

这里是一个修复方法,它不会随着时间的推移而增长:http://play.golang.org/p/GWQ2hJiySP

package main

import (
	"fmt"
	"sync"
	"time"
)

var datastore = make(map[int64]string)

var m sync.Mutex

func adder() {
	var count int64 = 0
	for {
		m.Lock()
		datastore[count] = "kjnbhjsdhgvsaghbsdbasjsabjhsabasbdjashdbashdbjasbdhasbdjbdjbdjhabjds"
		m.Unlock()
		count++
		time.Sleep(10 * time.Microsecond)

	}
}

func deleter() {
	for {
		m.Lock()
		for key, _ := range datastore {
			delete(datastore, key)
			time.Sleep(1 * time.Microsecond)
		}
		m.Unlock()
	}
}

func main() {
	// Start adding stuff to datastore (with MORE sleep time = 10 Microsecond)
	go adder()

	// Wait for some time
	time.Sleep(1 * time.Second)

	// Start deleting stuff from datastore (With LESS sleep time = 1 Microsecond)
	go deleter()

	time.Sleep(1 * time.Hour)
	fmt.Println("Done")
}

我在deleter中将锁放在了整个循环周围。实际上,只在delete周围放置锁就足够了,但是你不应该使用range来删除条目,因为修改map可能会改变迭代的顺序,所以这种结构不应该在实际程序中使用。

英文:

You are executing your deleter only once. In addition, map access is not concurrent-safe so you need to add Mutex around your code.

Here is a fix, that will not grow with time: http://play.golang.org/p/GWQ2hJiySP

package main

import (
	&quot;fmt&quot;
	&quot;sync&quot;
	&quot;time&quot;
)

var datastore = make(map[int64]string)

var m sync.Mutex

func adder() {
	var count int64 = 0
	for {
		m.Lock()
		datastore[count] = &quot;kjnbhjsdhgvsaghbsdbasjsabjhsabasbdjashdbashdbjasbdhasbdjbdjbdjhabjds&quot;
		m.Unlock()
		count++
		time.Sleep(10 * time.Microsecond)

	}
}

func deleter() {
	for {
		m.Lock()
		for key, _ := range datastore {
			delete(datastore, key)
			time.Sleep(1 * time.Microsecond)
		}
		m.Unlock()
	}
}

func main() {
	// Start adding stuff to datastore (with MORE sleep time = 10 Microsecond)
	go adder()

	// Wait for some time
	time.Sleep(1 * time.Second)

	// Start deleting stuff from datastore (With LESS sleep time = 1 Microsecond)
	go deleter()

	time.Sleep(1 * time.Hour)
	fmt.Println(&quot;Done&quot;)
}

I have put the lock in the deleter around the whole loop. It should be only necessary around the delete, however, you should not delete entries that way, with a range, because modifying the map can change the order of the iteration, so this structure should not be used in a real-life program.

huangapple
  • 本文由 发表于 2014年4月5日 21:31:12
  • 转载请务必保留本文链接:https://go.coder-hub.com/22881549.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定