英文:
What is correct way to use goroutine?
问题
我需要对每个请求应用一些测试,并根据测试结果触发响应。如果其中一个测试失败,我需要立即发送响应,否则我会等待所有测试成功完成。我希望使用并发来进行这些测试。
现在,我像这样做(简化):
func handler_request_checker(w http.ResponseWriter, r *http.Request) {
done := make(chan bool)
quit := make(chan bool)
counter := 0
go TestOne(r,done,quit)
go TestTwo(r,done,quit)
..............
go TestTen(r,done,quit)
for {
select {
case <- quit:
fmt.Println("收到退出信号")
return
case <- done:
counter++
if counter == 10 {
fmt.Println("所有检查都通过了")
return
}
}
}
}
func main() {
http.HandleFunc("/", handler_request_checker)
http.ListenAndServe()
}
一个goroutine的示例:
func TestOne(r *http.Request, done,quit chan bool) {
ip,_,ok := net.SplitHostPort(r.RemoteAddr)
if ok == nil {
for _,item := range BAD_IP_LIST {
if strings.Contains(ip,item) {
quit <- true
return
}
}
done <- true
return
} else {
quit <- true
return
}
}
问题是goroutine在收到退出信号后没有释放内存。我猜想这是因为done通道中有一些东西。
我对GO完全不熟悉,所以可能我使用它们的方式不对?
例如,当我启动负载检查http_load -parallel 10 -seconds 10
时,这个goroutine很容易吃掉100多MB的内存,并且不会将其归还给系统。在下一次检查时,它会再吃掉100多MB,依此类推。
如果我不使用go
(逐步进行测试),程序在任何负载检查下都不会超过10-15MB。
英文:
I need to apply some tests to each request and fire responce based on result of tests. If one of the test fail, I need to send responce imediatelly, otherwise I wait when all tests are done succesfully. I want to do that tests with concurrency.
Now, I do that like this (simplified):
func handler_request_checker(w http.ResponseWriter, r *http.Request) {
done := make(chan bool)
quit := make(chan bool)
counter := 0
go TestOne(r,done,quit)
go TestTwo(r,done,quit)
..............
go TestTen(r,done,quit)
for {
select {
case <- quit:
fmt.Println("got quit signal")
return
case <- done:
counter++
if counter == 10 {
fmt.Println("All checks passed succesfully")
return
}
}
}
func main() {
http.HandleFunc("/", handler_request_checker)
http.ListenAndServe()
}
Example of one of goroutine:
func TestOne(r *http.Request, done,quit chan bool) {
ip,_,ok := net.SplitHostPort(r.RemoteAddr)
if ok == nil {
for _,item := range BAD_IP_LIST {
if strings.Contains(ip,item) {
quit <- true
return
}
}
done <- true
return
} else {
quit <- true
return
}
}
Problem is that goroutines doesnt' free memory after I go t quit signal. I suppose that happens because there is something in done chanel.
I'm completely new in GO, so maybe I use them wrong way?
For example, when I start load check http_load -parallel 10 -seconds 10
, that goroutine easily eats 100+ MB of RAM and dont'g give it back to the system. On next check, it eat 100+ MB more, and so on.
If I do that tests without go
(step-by-step), program takes no more than 10-15mb with any load checks.
答案1
得分: 2
你的猜测是正确的。你正在使用同步通道,这意味着发送者和接收者都必须可用才能传输值。
一旦你的handler_request_checker
函数收到退出信号,它就停止检索done
和quit
通道的任何值。进一步的TestOne
、TestTwo
等goroutine在尝试发送它们的结果时将被阻塞。更糟糕的是,它们将永远停留在内存中,因为它们仍在运行,因为它们尚未传输它们的结果。
解决这个问题的一种方法是为done
和quit
使用带缓冲的(异步)通道。例如:
done := make(chan bool, 10)
quit := make(chan bool, 10)
如果你为你的10个测试使用缓冲区大小为10,那么所有的goroutine都能够发送它们的结果,即使没有读取器可用。所有的goroutine都将干净地退出,并且一旦所有的goroutine退出,包含一些未读结果的通道将被垃圾回收。
英文:
Your guess is right. You are using synchronous channels, which means that both the sender and the receiver must be available in order to transmit a value.
Once your handler_request_checker
function gets a quit signal, it stops retrieving any values of the done
and the quit
channel. Further TestOne
, TestTwo
, etc. goroutines will be blocked when they try to send their result. Even worse, they will stay in memory forever because they are still running since they haven't transmitted their result yet.
One way to solve the problem would be to use buffered (asynchronous) channels for done
and quit
. For example:
done := make(chan bool, 10)
quit := make(chan bool, 10)
If you use a buffer size of 10 for your 10 tests, then all goroutines are able to send their results, even when there is no reader available anymore. All goroutines will exit cleanly and the channels (that might contain some unread results) will get garbage collected once all goroutines have quit.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论