Golang并发代码审查的Codewalk

huangapple go评论79阅读模式
英文:

Golang Concurrency Code Review of Codewalk

问题

我正在尝试理解Golang并发的最佳实践。我读了O'Reilly关于Go并发的书,然后回到了Golang Codewalks,特别是这个例子:

https://golang.org/doc/codewalk/sharemem/

这是我希望与您一起审查的代码,以便更多地了解Go。我的第一印象是,这段代码违反了一些最佳实践。当然,这只是我(非常)没有经验的观点,我希望讨论并获得一些见解。这不是关于谁对谁错的问题,请友善一点,我只是想分享我的观点并获得一些反馈。也许这个讨论会帮助其他人看到我为什么错了,并教给他们一些东西。

我完全意识到这段代码的目的是教给初学者,而不是完美的代码。

问题1 - 没有Goroutine清理逻辑

func main() {
    // 创建输入和输出通道。
    pending, complete := make(chan *Resource), make(chan *Resource)

    // 启动StateMonitor。
    status := StateMonitor(statusInterval)

    // 启动一些Poller goroutine。
    for i := 0; i < numPollers; i++ {
        go Poller(pending, complete, status)
    }

    // 将一些资源发送到待处理队列。
    go func() {
        for _, url := range urls {
            pending <- &Resource{url: url}
        }
    }()

    for r := range complete {
        go r.Sleep(pending)
    }
}

main函数没有清理Goroutine的方法,这意味着如果这是一个库的一部分,它们将被泄漏。

问题2 - 写入者没有生成通道

我读到作为最佳实践,创建写入清理通道的逻辑应该由单个实体(或一组实体)控制。背后的原因是当写入者向关闭的通道写入时,会发生panic。因此,最好由写入者创建通道,向其写入并控制何时关闭它。如果有多个写入者,它们可以使用WaitGroup进行同步。

func StateMonitor(updateInterval time.Duration) chan<- State {
    updates := make(chan State)
    urlStatus := make(map[string]string)
    ticker := time.NewTicker(updateInterval)
    go func() {
        for {
            select {
            case <-ticker.C:
                logState(urlStatus)
            case s := <-updates:
                urlStatus[s.url] = s.status
            }
        }
    }()
    return updates
}

这个函数不应该负责创建updates通道,因为它是通道的读取者,而不是写入者。这个通道的写入者应该创建它并将其传递给这个函数。基本上是告诉函数“我将通过这个通道向你传递更新”。但是,这个函数正在创建一个通道,而不清楚谁负责清理它。

问题3 - 异步写入通道

这个函数:

func (r *Resource) Sleep(done chan<- *Resource) {
    time.Sleep(pollInterval + errTimeout*time.Duration(r.errCount))
    done <- r
}

在这里被引用:

for r := range complete {
    go r.Sleep(pending)
}

这似乎是一个糟糕的主意。当这个通道关闭时,我们将有一个goroutine在某个我们无法控制的地方休眠,等待写入该通道。假设这个goroutine休眠1小时,当它醒来时,它将尝试写入在清理过程中关闭的通道。这是另一个例子,说明通道的写入者应该负责清理过程。在这里,我们有一个完全自由并且不知道通道何时关闭的写入者。

请注意

如果我漏掉了代码中的任何问题(与并发有关),请列出它们。它不一定是客观问题,如果出于任何原因您希望以不同的方式设计代码,我也有兴趣了解。

从这段代码中得到的最重要的教训

对我来说,从审查这段代码中得到的最重要的教训是通道的清理和写入必须同步。它们必须在同一个for{}中,或者至少以某种方式进行通信(也许通过其他通道或原语),以避免向关闭的通道写入。

英文:

I'm trying to understand best practices for Golang concurrency. I read O'Reilly's book on Go's concurrency and then came back to the Golang Codewalks, specifically this example:

https://golang.org/doc/codewalk/sharemem/

This is the code I was hoping to review with you in order to learn a little bit more about Go. My first impression is that this code is breaking some best practices. This is of course my (very) unexperienced opinion and I wanted to discuss and gain some insight on the process. This isn't about who's right or wrong, please be nice, I just want to share my views and get some feedback on them. Maybe this discussion will help other people see why I'm wrong and teach them something.

I'm fully aware that the purpose of this code is to teach beginners, not to be perfect code.

Issue 1 - No Goroutine cleanup logic

func main() {
	// Create our input and output channels.
	pending, complete := make(chan *Resource), make(chan *Resource)

	// Launch the StateMonitor.
	status := StateMonitor(statusInterval)

	// Launch some Poller goroutines.
	for i := 0; i &lt; numPollers; i++ {
		go Poller(pending, complete, status)
	}

	// Send some Resources to the pending queue.
	go func() {
		for _, url := range urls {
			pending &lt;- &amp;Resource{url: url}
		}
	}()

	for r := range complete {
		go r.Sleep(pending)
	}
}

The main method has no way to cleanup the Goroutines, which means if this was part of a library, they would be leaked.

Issue 2 - Writers aren't spawning the channels

I read that as a best practice, the logic to create, write and cleanup a channel should be controlled by a single entity (or group of entities). The reason behind this is that writers will panic when writing to a closed channel. So, it is best for the writer(s) to create the channel, write to it and control when it should be closed. If there are multiple writers, they can be synced with a WaitGroup.

func StateMonitor(updateInterval time.Duration) chan&lt;- State {
	updates := make(chan State)
	urlStatus := make(map[string]string)
	ticker := time.NewTicker(updateInterval)
	go func() {
		for {
			select {
			case &lt;-ticker.C:
				logState(urlStatus)
			case s := &lt;-updates:
				urlStatus[s.url] = s.status
			}
		}
	}()
	return updates
}

This function shouldn't be in charge of creating the updates channel because it is the reader of the channel, not the writer. The writer of this channel should create it and pass it to this function. Basically saying to the function "I will pass updates to you via this channel". But instead, this function is creating a channel and it isn't clear who is responsible of cleaning it up.

Issue 3 - Writing to a channel asynchronously

This function:

func (r *Resource) Sleep(done chan&lt;- *Resource) {
	time.Sleep(pollInterval + errTimeout*time.Duration(r.errCount))
	done &lt;- r
}

Is being referenced here:

for r := range complete {
    go r.Sleep(pending)
}

And it seems like an awful idea. When this channel is closed, we'll have a goroutine sleeping somewhere out of our reach waiting to write to that channel. Let's say this goroutine sleeps for 1h, when it wakes up, it will try to write to a channel that was closed in the cleanup process. This is another example of why the writters of the channels should be in charge of the cleanup process. Here we have a writer who's completely free and unaware of when the channel was closed.

Please

If I missed any issues from that code (related to concurrency), please list them. It doesn't have to be an objective issue, if you'd have designed the code in a different way for any reason, I'm also interested in learning about it.

Biggest lesson from this code

For me the biggest lesson I take from reviewing this code is that the cleanup of channels and the writing to them has to be synchronized. They have to be in the same for{} or at least communicate somehow (maybe via other channels or primitives) to avoid writing to a closed channel.

答案1

得分: 2

  1. 这是main方法,所以不需要清理。当main返回时,程序退出。如果这不是main方法,那么你是正确的。

  2. 没有适用于所有用例的最佳实践。你在这里展示的代码是一种非常常见的模式。该函数创建了一个goroutine,并返回一个通道,以便其他人可以与该goroutine进行通信。没有规定通道必须如何创建的规则。然而,没有办法终止该goroutine。这种模式非常适用于从数据库中读取大型结果集的情况。通道允许在从数据库读取数据时进行数据流传输。在这种情况下,通常还有其他方式来终止goroutine,比如传递一个上下文。

  3. 同样,没有硬性规定通道应该如何创建/关闭。通道可以保持打开状态,当不再使用时,它将被垃圾回收。如果用例要求如此,通道可以无限期地保持打开状态,你担心的情况将永远不会发生。

英文:
  1. It is the main method, so there is no need to cleanup. When main returns, the program exits. If this wasn't the main, then you would be correct.

  2. There is no best practice that fits all use cases. The code you show here is a very common pattern. The function creates a goroutine, and returns a channel so that others can communicate with that goroutine. There is no rule that governs how channels must be created. There is no way to terminate that goroutine though. One use case this pattern fits well is reading a large resultset from a
    database. The channel allows streaming data as it is read from the
    database. In that case usually there are other means of terminating the
    goroutine though, like passing a context.

  3. Again, there are no hard rules on how channels should be created/closed. A channel can be left open, and it will be garbage collected when it is no longer used. If the use case demands so, the channel can be left open indefinitely, and the scenario you worry about will never happen.

答案2

得分: 2

  1. 你问的是这段代码是否是库的一部分,是的,在库函数中创建没有清理的goroutine是一种不好的做法。如果这些goroutine执行的是库的文档化行为,那么问题在于调用者不知道这个行为何时发生。如果有一些通常是“fire and forget”(创建后就忘记)的行为,应该由调用者决定何时忘记它。例如:
func doAfter5Minutes(f func()) {
   go func() {
       time.Sleep(5 * time.Minute)
       f()
       log.Println("done!")
   }()
}

有道理,对吧?当你调用这个函数时,它会在5分钟后执行某个操作。问题是,很容易误用这个函数,比如这样:

// 每5分钟执行一次重要任务
for {
    doAfter5Minutes(importantTaskFunction)
}

乍一看,这似乎没问题。我们每5分钟执行一次重要任务,对吧?实际上,我们很快就会创建很多goroutine,可能在它们开始消耗所有可用内存之前就会耗尽内存。

我们可以实现某种回调或通道来在任务完成时发出信号,但实际上,这个函数应该简化如下:

func doAfter5Minutes(f func()) {
   time.Sleep(5 * time.Minute)
   f()
   log.Println("done!")
}

现在调用者可以选择如何使用它:

// 同步调用
doAfter5Minutes(importantTaskFunction)
// 创建并忘记
go doAfter5Minutes(importantTaskFunction)
  1. 这个函数可能也需要改变。正如你所说,写入者应该拥有通道,因为他们应该负责关闭它。这个通道读取函数坚持要创建它读取的通道,实际上迫使自己进入了上面提到的不好的“fire and forget”模式。注意函数需要从通道读取,但在读取之前需要返回通道。因此,它必须将读取行为放在一个新的、未管理的goroutine中,以便立即返回通道。
func StateMonitor(updates chan State, updateInterval time.Duration) {
    urlStatus := make(map[string]string)
    ticker := time.NewTicker(updateInterval)
    defer ticker.Stop() // 不停止ticker也是资源泄漏

    for {
        select {
        case <-ticker.C:
            logState(urlStatus)
        case s := <-updates:
            urlStatus[s.url] = s.status
        }
    }

}

注意,这个函数现在更简单、更灵活和同步。前一个版本真正实现的唯一一件事是,它(大部分)保证每个StateMonitor实例都有一个独立的通道,而不会出现多个监视器竞争同一个通道的读取的情况。虽然这可能有助于避免某类错误,但它也使函数变得不太灵活,并且更容易出现资源泄漏。

  1. 我不确定我是否真正理解这个例子,但是关于通道关闭的黄金法则是写入者应该始终负责关闭通道。记住这个规则,并注意一下这段代码的几个要点:
  • Sleep方法向r写入数据
  • Sleep方法并发执行,没有跟踪运行的实例数量、状态等的方法。

仅根据这些要点,我们可以说在程序中可能没有任何地方是安全关闭r的,因为似乎没有办法知道它是否会被再次使用。

英文:
  1. As you are asking about if this code was part of a library, yes it would be poor practice to spawn goroutines with no cleanup inside a library function. If those goroutines carry out documented behaviour of the library, it's problematic that the caller doesn't know when that behaviour is going to happen. If you have any behaviour that is typically "fire and forget", it should be the caller who chooses when to forget about it. For example:
func doAfter5Minutes(f func()) {
   go func() {
       time.Sleep(5 * time.Minute)
       f()
       log.Println(&quot;done!&quot;)
   }()
}

Makes sense, right? When you call the function, it does something 5 minutes later. The problem is that it's easy to misuse this function like this:

// do the important task every 5 minutes
for {
    doAfter5Minutes(importantTaskFunction)
}

At first glance, this might seem fine. We're doing the important task every 5 minutes, right? In reality, we're spawning many goroutines very quickly, probably consuming all available memory before they start dropping off.

We could implement some kind of callback or channel to signal when the task is done, but really, the function should be simplified like so:

func doAfter5Minutes(f func()) {
   time.Sleep(5 * time.Minute)
   f()
   log.Println(&quot;done!&quot;)
}

Now the caller has the choice of how to use it:

// call synchronously
doAfter5Minutes(importantTaskFunction)
// fire and forget
go doAfter5Minutes(importantTaskFunction)
  1. This function arguably should also be changed. As you say, the writer should effectively own the channel, as they should be the one closing it. The fact that this channel-reading function insists on creating the channel it reads from actually coerces itself into this poor "fire and forget" pattern mentioned above. Notice how the function needs to read from the channel, but it also needs to return the channel before reading. It therefore had to put the reading behaviour in a new, un-managed goroutine to allow itself to return the channel right away.
func StateMonitor(updates chan State, updateInterval time.Duration) {
    urlStatus := make(map[string]string)
    ticker := time.NewTicker(updateInterval)
    defer ticker.Stop() // not stopping the ticker is also a resource leak

    for {
        select {
        case &lt;-ticker.C:
            logState(urlStatus)
        case s := &lt;-updates:
            urlStatus[s.url] = s.status
        }
    }

}

Notice that the function is now simpler, more flexible and synchronous. The only thing that the previous version really accomplishes, is that it (mostly) guarantees that each instance of StateMonitor will have a channel all to itself, and you won't have a situation where multiple monitors are competing for reads on the same channel. While this may help you avoid a certain class of bugs, it also makes the function a lot less flexible and more likely to have resource leaks.

  1. I'm not sure I really understand this example, but the golden rule for channel closing is that the writer should always be responsible for closing the channel. Keep this rule in mind, and notice a few points about this code:
  • The Sleep method writes to r
  • The Sleep method is executed concurrently, with no method of tracking how many instances are running, what state they are in, etc.

Based on these points alone, we can say that there probably isn't anywhere in the program where it would be safe to close r, because there's seemingly no way of knowing if it will be used again.

huangapple
  • 本文由 发表于 2021年6月15日 10:37:31
  • 转载请务必保留本文链接:https://go.coder-hub.com/67979304.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定