英文:
Golang channels make program wait indefinitely
问题
我正在处理一个具有管理者/工作者风格设置的程序。它有n
个工作者,从一个HTTP源获取数据,将数据推送到gameData通道,从urlData通道中获取URL信息并重复执行。然后,我有一个goroutine,它应该从从gameData通道获取数据的API中解析数据,并将URL推送到urlData通道。
我的问题是,当任一goroutine第二次尝试将某些内容推送到其中一个通道时,goroutines会停止,并且我必须终止程序。
以下是一些代码和存储库链接:https://github.com/gaigepr/lolTeams
编辑:
在阅读有关使用缓冲通道的评论后,问题显然是死锁。然而,使通道缓冲只会使死锁发生的时间更长。
此时,同步解析器和getgames线程是确保没有死锁的正确方法吗?
编辑2:
我重构了程序,现在我正在使用线程安全队列来存储召唤师ID,每个工作者线程现在都在自己的数据上调用解析器函数。这种简化了事情。我这样做是因为我无法解决playerChan最终因数据累积的性质而填满的问题。除非有人有理由不这样做,否则我将保持这个问题的开放状态,看看是否有人有解决通道问题的答案。
英文:
I am working on a program with a manager/worker style setup. It has n
number of workers that get data from an http source, push the data onto a gameData channel, pull url information off of a urlData channel and repeat. I then have one goroutine that is supposed to be parsing the data from the api that pulls from the channel gameData and pushes urls onto urlData channel.
My problem is that the second time either goroutine tried to push something onto one of the channels, the goroutines halt and I have to kill the program.
Here is some of the code and a link to the repo: https://github.com/gaigepr/lolTeams
EDIT:
After reading the comments about using buffered channels, the problem is obviously a deadlock. However, making the channels buffered just makes the deadlock take longer to happen.
At this point is synchronizing the parser and getgames threads the way to go to ensure no deadlocks?
EDIT2:
I refactored the program so that now I am using a threadsafe queue to store the summonerID as well as each worker thread now calls the parser function on its own data. This simplified things in a good way. I did this because I was unable to work around the playerChan eventually filling up because of the nature of the data accumulating. unless someone has a reason not to, I will leave this question open to see if someone has an answer that solves to channel problem.
答案1
得分: 2
我还没有仔细研究你的长代码片段,但我怀疑你在向一个通道推送数据而没有从中读取,所以通道会填满,最终你的推送操作将无限期地被阻塞。
如果你想得到更好的帮助,我建议你发布一个简单的程序(大约20行),以便我们可以在http://golang.org上运行它,从而重现这个问题。
英文:
I haven't studied your long code fragment carefully but I suspect that you are pushing to a channel without reading from it, so the channel will fill up and eventually your pushes will start blocking indefinitely.
If you want better help, I suggest posting a simple program (~20 lines) that reproduces the problem so that we can run it for ourselves on http://golang.org.
答案2
得分: 2
让我给你提个建议,你可以使用default
情况,例如:
default:
continue
这样可以帮助你处理在之前指定的通道中没有传入值的情况,所以你可以简单地继续循环,直到通道中有值传入,或者你可以使用类似于以下的超时情况:
case <-time.After(6 * time.Second):
continue
当然,这两种情况都允许你继续循环遍历通道,从技术上讲,除非其他Go协程停止通过通道发送值,否则你不会遇到死锁问题。
英文:
let me suggest something, you can either use the default
case e.g.
default:
continue
which helps you handle cases when there are no incoming values from channel(s) you've specified earlier, so you can simply continue your for loop until something comes down the channel, or you could use a timeout case like this:
case <-time.After(6 * time.Second):
continue
Of course these 2 cases will allow you to continue cycling through the channels and technically you wont experience a deadlock unless other go-routines stop sending values through the channels.
答案3
得分: 2
这两个操作可能会互相等待并导致死锁:
gameChan <- game
playerChan <- summonerID
可以通过在其中一个goroutine中放置一个切片(或链表)来存储累积数据来解决这个问题。将通道设置为缓冲区不会有帮助,因为它受大小限制。
类似于这样:http://play.golang.org/p/ZtiPkEoPI6
英文:
Those two operations may wait for each other and cause the deadlock:
gameChan<-game
playerChan<-summonerID
This can be solved by put a slice (or a link list) into one of those go routines to store the accumulating data. Make channel buffered won’t help because it’s limit by size.
Something like this: http://play.golang.org/p/ZtiPkEoPI6
答案4
得分: 0
问题出在我收集数据的方式上。最终,playerChan会填满,导致死锁。最后,我进行了重构,并使用了线程安全的队列,效果非常好。
英文:
The problem was with the nature of the data I was collecting. Eventually the playerChan would fill up resulting deadlocks. I ended up refactoring and using threadsafe queues which worked out quite nicely.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论