Go GC 停止了我的 goroutine 吗?

huangapple go评论82阅读模式
英文:

Go GC stopping my goroutine?

问题

我一直在尝试从传统的语言(如Java和C)转向Go语言,到目前为止,我一直在享受Go所提供的经过深思熟虑的设计选择。然而,当我开始我的第一个“真正”的项目时,我遇到了一个几乎没有人遇到的问题。

我的项目是一个简单的网络实现,用于发送和接收数据包。一般的结构如下(当然是简化的):

一个客户端通过net.Conn与服务器进行管理。这个Client创建了一个PacketReader和一个PacketWriter。这两个都在不同的goroutine中运行无限循环。PacketReader接收一个带有单个OnPacketReceived函数的接口,由客户端实现。

PacketReader的代码大致如下:

go func() {
    for {
        bytes, err := reader.ReadBytes(10) // 阻塞当前routine,直到有可用的字节。
        if err != nil {
            panic(err) // 处理错误
        }
        reader.handler.OnPacketReceived(reader.parseBytes(bytes))
    }
}()

PacketWriter的代码大致如下:

go func() {
    for {
        if len(reader.packetQueue) > 0 {
            // 写入数据包
        }
    }
}()

为了使Client阻塞,客户端创建了一个由OnPacketReceived填充的通道,类似于以下代码:

type Client struct {
    callbacks map[int]chan interface{}
    // 更多字段
}

func (c *Client) OnPacketReceived(packet *Packet) {
    c.callbacks[packet.Id] <- packet.Data
}

func (c *Client) SendDataBlocking(id int, data interface{}) interface{} {
    c.PacketWriter.QueuePacket(data)
    return <-c.callbacks[id]
}

现在我遇到的问题是:reader.parseBytes函数执行了一些密集的解码操作,创建了相当多的对象(以至于GC决定运行)。然而,GC会暂停当前正在解码字节的reader goroutine,然后挂起。类似于我的问题的问题在这里有描述。我已经确认确实是GC引起的问题,因为使用GOGC=off可以成功运行。

此时,我的三个routine的状态如下:

  • Client(主routine):等待通道
  • Writer:仍在运行,等待队列中的新数据
  • Reader:设置为可运行,但实际上没有运行

不知何故,GC要么无法停止所有routine以进行运行,要么在停止它们后无法恢复这些goroutine。

所以我的问题是:**有没有办法解决这个问题?**我对Go还不太熟悉,所以我不知道我的设计选择是否符合常规,我可以改变程序的结构。我是否需要改变处理数据包读取回调的方式,或者尝试减少数据包解码的密集程度?谢谢!

编辑:我正在运行Go 1.5.1,我将尝试在今天晚些时候提供一个可工作的示例。

英文:

I have been trying to get into Go from the more traditional languages such as Java and C and so far I've been enjoying the well-thought out design choices that Go offers. When I started my first "real" project though, I ran into a problem that almost nobody seems to have.

My project is a simple networking implementation that sends and receives packets. The general structure is something like this (of course simplified):

A client manages the net.Conn with the server. This Clientcreates a PacketReaderand a PacketWriter. These both run infinite loops in a different goroutine. The PacketReader takes an interface with a single OnPacketReceived function that is implemented by the client.

The PacketReader code looks something like this:

go func() {
    for {
        bytes, err := reader.ReadBytes(10) // Blocks the current routine until bytes are available.
        if err != nil {
            panic(err) // Handle error
        }
        reader.handler.OnPacketReceived(reader.parseBytes(bytes))
    }
}()

The PacketWriter code looks something like this:

go func() {

    for {
        if len(reader.packetQueue) &gt; 0 {
            // Write packet
        }
    }
}()

In order to make Client blocking, the client makes a channel that gets filled by OnPacketReceived, something like this:

type Client struct {
    callbacks map[int]chan interface{}
    // More fields
}

func (c *Client) OnPacketReceived(packet *Packet) {
    c.callbacks[packet.Id] &lt;- packet.Data
}

func (c *Client) SendDataBlocking(id int, data interface{}) interface{} {
    c.PacketWriter.QueuePacket(data)
    return &lt;-c.callbacks[id]
}

Now here is my problem: the reader.parseBytes function does some intensive decoding operation that creates quite a lot of objects (to the point that the GC decides to run). The GC however, pauses the reader goroutine that is currently decoding the bytes, and then hangs. A problem that seems similar to mine is described here. I have confirmed that it is actually the GC causing it, because running it with GOGC=off runs successfully.

At this point, my 3 routines look like this:

  • Client (main routine): Waiting for channel
  • Writer: Still running, waiting for new data in queue
  • Reader: Set as runnable, but not actually running

Somehow, the GC is either not able to stop all routines in order to run, or it does not resume said goroutines after it stopped them.

So my question is this: Is there any way to fix this? I am new to Go so I don't really know if my design choices are even remotely conventional, and I'm all up with changing the structure of my program. Do I need to change the way I handle packet reading callbacks, do I need to try and make the packet decoder less intensive? Thanks!

Edit: I am running Go 1.5.1, I'll try to get a working example later today.

答案1

得分: 2

根据mrd0ll4r的评论,将写入器更改为使用通道而不是切片(我甚至不知道为什么一开始要这样做)。这似乎给了垃圾回收足够的“灵活性”来允许线程停止。添加runtime.Gosched()并继续使用切片也可以工作,但通道似乎更符合Go语言的风格。

英文:

As per mrd0ll4rs comment, changed the writer to use channels instead of a slice (I don't even know why I did that in the first place). That seemed to give the GC enough "mobility" to allow the threads to stop. Adding in the runtime.Gosched() and still using slices also worked though, but the channels seemed more "go-esque".

huangapple
  • 本文由 发表于 2015年10月27日 17:28:41
  • 转载请务必保留本文链接:https://go.coder-hub.com/33364138.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定