英文:
In golang, why does my program run slower when I use a buffered (asynchronous) channel?
问题
我对golang还比较新，所以我相信这个问题在很大程度上是由于我对一些概念上的不足造成的。
在golang中，我们可以有两种类型的通道：无缓冲和有缓冲（同步和异步）。
unbufferedChan := make(chan string)
bufferedChan := make(chan string, 100)
通过无缓冲通道进行通信的两个goroutine必须互相等待。也就是说，接收goroutine在发送方发送之前会阻塞，而发送方在接收方接收之前会阻塞。
在有缓冲的情况下，只有当通道为空时接收方才会阻塞。只有当通道已满时发送方才会阻塞。
通过使用有缓冲的通道，我期望减少goroutine阻塞的时间，从而提高代码的速度。然而，事实并非如此。与我的期望相反（即“有缓冲更高效”），无缓冲通道实际上更快。
这是所有的代码。你需要先go get code.google.com/p/go-html-transform/html/transform。
package main
import (
"fmt"
"math/rand"
"os"
"sync"
"time"
sel "code.google.com/p/go-html-transform/css/selector"
h5 "code.google.com/p/go-html-transform/h5"
gnhtml "code.google.com/p/go.net/html"
)
// 找到特定的HTML元素并返回其文本元素子节点。
func main() {
test := `
<html>
<head>
<title>This is the test document!</title>
<style>
header: color=blue;
</style>
</head>
<body>
<div id="h" class="header">This is some text</div>
</body>
</html>`
// 获取此HTML的解析树
h5tree, err := h5.NewFromString(test)
if err != nil {
die(err)
}
n := h5tree.Top()
// 从CSS选择器语句创建一个Chain对象
chn, err := sel.Selector("#h")
if err != nil {
die(err)
}
// 找到该项。应该是一个带有文本“This is some text”的div节点
h := chn.Find(n)[0]
// 运行我们的小实验这么多次
var iter int = 100000
// 当缓冲时，缓冲区大小应该是多少？
var bufSize uint = 100
// 记录我们尝试使用缓冲和无缓冲通道的次数
var bufCount int = 0
var unbufCount int =0
// 记录经过的纳秒数的总和
var bufSum int64 = 0
var unbufSum int64 = 0
// 调用函数{iter}次，随机选择使用缓冲或无缓冲通道。
for i := 0; i < iter; i++ {
if rand.Float32() < 0.5 {
// 无缓冲
unbufCount += 1
startTime := time.Now()
getAllText(h, 0)
unbufSum += time.Since(startTime).Nanoseconds()
} else {
// 使用缓冲
bufCount += 1
startTime := time.Now()
getAllText(h, bufSize)
bufSum += time.Since(startTime).Nanoseconds()
}
}
unbufAvg := unbufSum / int64(unbufCount)
bufAvg := bufSum / int64(bufCount)
fmt.Printf("无缓冲平均时间（纳秒）：%v\n", unbufAvg)
fmt.Printf("缓冲平均时间（纳秒）：%v\n", bufAvg)
}
// 终止程序并报告错误
func die(err error) {
fmt.Printf("终止：%v\n", err.Error())
os.Exit(1)
}
// 遍历所有节点的子节点，并构建一个由c.Data组成的字符串，其中c.Type == TextNode
func getAllText(n *gnhtml.Node, bufSize uint) string {
var texts chan string
if bufSize == 0 {
// 无缓冲，同步
texts = make(chan string)
} else {
// 有缓冲，异步
texts = make(chan string, bufSize)
}
wg := sync.WaitGroup{}
// Go遍历所有n的子节点，只发送文本数据到texts通道。
wg.Add(1)
nTree := h5.NewTree(n)
go func () {
nTree.Walk(func (c *gnhtml.Node) {
if c.Type == gnhtml.TextNode {
texts <- c.Data
}
})
close(texts)
wg.Done()
}()
// 当文本数据通过texts通道传入时，构建finalString
wg.Add(1)
finalString := ""
go func () {
for t := range texts {
finalString += t
}
wg.Done()
}()
// 两个goroutine都完成后，返回finalString。
wg.Wait()
return finalString
}
运行程序10次，以下是结果：
缓冲平均时间（纳秒）：32088
缓冲平均时间（纳秒）：32183
缓冲平均时间（纳秒）：35091
缓冲平均时间（纳秒）：35798
缓冲平均时间（纳秒）：36966
缓冲平均时间（纳秒）：38707
缓冲平均时间（纳秒）：39464
缓冲平均时间（纳秒）：40021
缓冲平均时间（纳秒）：41063
缓冲平均时间（纳秒）：46248
无缓冲平均时间（纳秒）：18265
无缓冲平均时间（纳秒）：18804
无缓冲平均时间（纳秒）：20268
无缓冲平均时间（纳秒）：20401
无缓冲平均时间（纳秒）：21652
无缓冲平均时间（纳秒）：22630
无缓冲平均时间（纳秒）：22907
无缓冲平均时间（纳秒）：23326
无缓冲平均时间（纳秒）：24133
无缓冲平均时间（纳秒）：27546
英文:
I'm fairly new to golang, so I'm sure that this question is largely due to some conceptual shortcomings on my part.
In golang, we can have two types of channels: unbuffered and buffered (synchronous and asynchronous, respectively).
unbufferedChan := make(chan string)
bufferedChan := make(chan string, 100)
Two goroutines communicating via an unbuffered channel must wait on each other. That is, the receiving goroutine blocks until the sender sends, and the sender blocks until the receiver receives.
In the buffered case, the receiver blocks only if the channel is empty. The sender blocks only if the channel is full.
By using the buffered channel, I expected to decrease the amount of time the goroutines were blocking, and therefore increase the speed of the code. Instead, it all went slower. Contrary to my expectations (i.e., "Buffered is more efficient") the unbuffered channel ended up being substantially faster.
Here's all the code. You'll need to go get code.google.com/p/go-html-transform/html/transform first.
package main
import (
"fmt"
"math/rand"
"os"
"sync"
"time"
sel "code.google.com/p/go-html-transform/css/selector"
h5 "code.google.com/p/go-html-transform/h5"
gnhtml "code.google.com/p/go.net/html"
)
// Find a specific HTML element and return its textual element children.
func main() {
test := `
<html>
<head>
<title>This is the test document!</title>
<style>
header: color=blue;
</style>
</head>
<body>
<div id="h" class="header">This is some text</div>
</body>
</html>`
// Get a parse tree for this HTML
h5tree, err := h5.NewFromString(test)
if err != nil {
die(err)
}
n := h5tree.Top()
// Create a Chain object from a CSS selector statement
chn, err := sel.Selector("#h")
if err != nil {
die(err)
}
// Find the item. Should be a div node with the text "This is some text"
h := chn.Find(n)[0]
// run our little experiment this many times total
var iter int = 100000
// When buffering, how large shall the buffer be?
var bufSize uint = 100
// Keep a running total of the number of times we've tried buffered
// and unbuffered channels.
var bufCount int = 0
var unbufCount int =0
// Keep a running total of the number of nanoseconds that have gone by.
var bufSum int64 = 0
var unbufSum int64 = 0
// Call the function {iter} times, randomly choosing whether to use a
// buffered or unbuffered channel.
for i := 0; i < iter; i++ {
if rand.Float32() < 0.5 {
// No buffering
unbufCount += 1
startTime := time.Now()
getAllText(h, 0)
unbufSum += time.Since(startTime).Nanoseconds()
} else {
// Use buffering
bufCount += 1
startTime := time.Now()
getAllText(h, bufSize)
bufSum += time.Since(startTime).Nanoseconds()
}
}
unbufAvg := unbufSum / int64(unbufCount)
bufAvg := bufSum / int64(bufCount)
fmt.Printf("Unbuffered average time (ns): %v\n", unbufAvg)
fmt.Printf("Buffered average time (ns): %v\n", bufAvg)
}
// Kill the program and report the error
func die(err error) {
fmt.Printf("Terminating: %v\n", err.Error())
os.Exit(1)
}
// Walk through all of a nodes children and construct a string consisting
// of c.Data where c.Type == TextNode
func getAllText(n *gnhtml.Node, bufSize uint) string {
var texts chan string
if bufSize == 0 {
// unbuffered, synchronous
texts = make(chan string)
} else {
// buffered, asynchronous
texts = make(chan string, bufSize)
}
wg := sync.WaitGroup{}
// Go walk through all n's child nodes, sending only textual data
// over the texts channel.
wg.Add(1)
nTree := h5.NewTree(n)
go func () {
nTree.Walk(func (c *gnhtml.Node) {
if c.Type == gnhtml.TextNode {
texts <- c.Data
}
})
close(texts)
wg.Done()
}()
// As text data comes in over the texts channel, build up finalString
wg.Add(1)
finalString := ""
go func () {
for t := range texts {
finalString += t
}
wg.Done()
}()
// Return finalString once both of the goroutines have finished.
wg.Wait()
return finalString
}
Running the program 10 times, here are the results:
Buffered average time (ns): 32088
Buffered average time (ns): 32183
Buffered average time (ns): 35091
Buffered average time (ns): 35798
Buffered average time (ns): 36966
Buffered average time (ns): 38707
Buffered average time (ns): 39464
Buffered average time (ns): 40021
Buffered average time (ns): 41063
Buffered average time (ns): 46248
Unbuffered average time (ns): 18265
Unbuffered average time (ns): 18804
Unbuffered average time (ns): 20268
Unbuffered average time (ns): 20401
Unbuffered average time (ns): 21652
Unbuffered average time (ns): 22630
Unbuffered average time (ns): 22907
Unbuffered average time (ns): 23326
Unbuffered average time (ns): 24133
Unbuffered average time (ns): 27546
答案1
得分: 3
实际上，我可能已经找到了答案。我认为额外的时间是由于创建了一个100个元素的缓冲通道。
我再次运行了程序，这次使用了一个大小为10的缓冲区（仍然比我的小测试程序所需的要大），而不是100（原始缓冲区大小）。
以下是结果：
缓冲平均时间（纳秒）：21930
缓冲平均时间（纳秒）：22721
缓冲平均时间（纳秒）：23011
缓冲平均时间（纳秒）：23707
缓冲平均时间（纳秒）：27701
缓冲平均时间（纳秒）：28325
缓冲平均时间（纳秒）：28851
缓冲平均时间（纳秒）：29641
缓冲平均时间（纳秒）：30417
缓冲平均时间（纳秒）：32600
无缓冲平均时间（纳秒）：21077
无缓冲平均时间（纳秒）：21490
无缓冲平均时间（纳秒）：22332
无缓冲平均时间（纳秒）：22584
无缓冲平均时间（纳秒）：26438
无缓冲平均时间（纳秒）：26824
无缓冲平均时间（纳秒）：27322
无缓冲平均时间（纳秒）：27926
无缓冲平均时间（纳秒）：27985
无缓冲平均时间（纳秒）：30322
这些数字彼此之间更接近。
英文:
Actually, I may have just figured it out. I think the extra time is due to the creation of a 100-element buffered channel.
I ran the program again, this time using a buffer size of 10 (still more than my little test program would need) instead of 100 (the original buffer size.)
Here are the results:
Buffered average time (ns): 21930
Buffered average time (ns): 22721
Buffered average time (ns): 23011
Buffered average time (ns): 23707
Buffered average time (ns): 27701
Buffered average time (ns): 28325
Buffered average time (ns): 28851
Buffered average time (ns): 29641
Buffered average time (ns): 30417
Buffered average time (ns): 32600
Unbuffered average time (ns): 21077
Unbuffered average time (ns): 21490
Unbuffered average time (ns): 22332
Unbuffered average time (ns): 22584
Unbuffered average time (ns): 26438
Unbuffered average time (ns): 26824
Unbuffered average time (ns): 27322
Unbuffered average time (ns): 27926
Unbuffered average time (ns): 27985
Unbuffered average time (ns): 30322
These numbers are much closer to each other.
答案2
得分: 2
在你的测试中，将缓冲区的创建外部化：
package main
import (
"fmt"
"math/rand"
"os"
"sync"
"time"
sel "code.google.com/p/go-html-transform/css/selector"
h5 "code.google.com/p/go-html-transform/h5"
gnhtml "code.google.com/p/go.net/html"
)
// 查找特定的HTML元素并返回其文本元素子节点。
func main() {
test := `
<html>
<head>
<title>This is the test document!</title>
<style>
header: color=blue;
</style>
</head>
<body>
<div id="h" class="header">This is some text</div>
</body>
</html>`
// 获取此HTML的解析树
h5tree, err := h5.NewFromString(test)
if err != nil {
die(err)
}
n := h5tree.Top()
// 从CSS选择器语句创建Chain对象
chn, err := sel.Selector("#h")
if err != nil {
die(err)
}
// 查找项目。应该是一个带有文本“This is some text”的div节点
h := chn.Find(n)[0]
// 运行我们的小实验这么多次
var iter int = 100000
// 缓冲时，缓冲区大小应该是多少？
var bufSize uint = 100
// 保持我们尝试使用缓冲和非缓冲通道的次数的总和。
var bufCount int = 0
var unbufCount int =0
// 保持经过的纳秒数的总和。
var bufSum int64 = 0
var unbufSum int64 = 0
// 调用函数{iter}次，随机选择使用缓冲或非缓冲通道。
for i := 0; i < iter; i++ {
if rand.Float32() < 0.5 {
// 无缓冲
unbufCount += 1
texts := make(chan string)
startTime := time.Now()
getAllText(h, 0, texts)
unbufSum += time.Since(startTime).Nanoseconds()
} else {
// 使用缓冲
bufCount += 1
texts := make(chan string, bufSize)
startTime := time.Now()
getAllText(h, bufSize, texts)
bufSum += time.Since(startTime).Nanoseconds()
}
}
unbufAvg := unbufSum / int64(unbufCount)
bufAvg := bufSum / int64(bufCount)
fmt.Printf("无缓冲平均时间（纳秒）：%v\n", unbufAvg)
fmt.Printf("缓冲平均时间（纳秒）：%v\n", bufAvg)
}
// 终止程序并报告错误
func die(err error) {
fmt.Printf("终止：%v\n", err.Error())
os.Exit(1)
}
// 遍历所有节点的子节点，并构造一个由c.Data组成的字符串，其中c.Type == TextNode
func getAllText(n *gnhtml.Node, bufSize uint, texts chan string) string {
wg := sync.WaitGroup{}
// 遍历n的所有子节点，仅通过texts通道发送文本数据。
wg.Add(1)
nTree := h5.NewTree(n)
go func () {
nTree.Walk(func (c *gnhtml.Node) {
if c.Type == gnhtml.TextNode {
texts <- c.Data
}
})
close(texts)
wg.Done()
}()
// 当文本数据通过texts通道传入时，构建finalString
wg.Add(1)
finalString := ""
go func () {
for t := range texts {
finalString += t
}
wg.Done()
}()
// 等待两个goroutine都完成后返回finalString。
wg.Wait()
return finalString
}
通过这种方式（在启动计时器之前移动通道创建），我得到了以下结果：
缓冲平均时间（纳秒）：2649
缓冲平均时间（纳秒）：2655
缓冲平均时间（纳秒）：2657
缓冲平均时间（纳秒）：2695
缓冲平均时间（纳秒）：2695
缓冲平均时间（纳秒）：2699
缓冲平均时间（纳秒）：2719
缓冲平均时间（纳秒）：2724
缓冲平均时间（纳秒）：2793
缓冲平均时间（纳秒）：3101
无缓冲平均时间（纳秒）：2997
无缓冲平均时间（纳秒）：3005
无缓冲平均时间（纳秒）：3027
无缓冲平均时间（纳秒）：3031
无缓冲平均时间（纳秒）：3048
无缓冲平均时间（纳秒）：3062
无缓冲平均时间（纳秒）：3086
无缓冲平均时间（纳秒）：3092
无缓冲平均时间（纳秒）：3095
无缓冲平均时间（纳秒）：3191
为了改进，可能的解决方案是在可能的情况下保持通道打开。
英文:
Externalize the creation of the buffer in your test :
package main
import (
"fmt"
"math/rand"
"os"
"sync"
"time"
sel "code.google.com/p/go-html-transform/css/selector"
h5 "code.google.com/p/go-html-transform/h5"
gnhtml "code.google.com/p/go.net/html"
)
// Find a specific HTML element and return its textual element children.
func main() {
test := `
<html>
<head>
<title>This is the test document!</title>
<style>
header: color=blue;
</style>
</head>
<body>
<div id="h" class="header">This is some text</div>
</body>
</html>`
// Get a parse tree for this HTML
h5tree, err := h5.NewFromString(test)
if err != nil {
die(err)
}
n := h5tree.Top()
// Create a Chain object from a CSS selector statement
chn, err := sel.Selector("#h")
if err != nil {
die(err)
}
// Find the item. Should be a div node with the text "This is some text"
h := chn.Find(n)[0]
// run our little experiment this many times total
var iter int = 100000
// When buffering, how large shall the buffer be?
var bufSize uint = 100
// Keep a running total of the number of times we've tried buffered
// and unbuffered channels.
var bufCount int = 0
var unbufCount int =0
// Keep a running total of the number of nanoseconds that have gone by.
var bufSum int64 = 0
var unbufSum int64 = 0
// Call the function {iter} times, randomly choosing whether to use a
// buffered or unbuffered channel.
for i := 0; i < iter; i++ {
if rand.Float32() < 0.5 {
// No buffering
unbufCount += 1
texts := make(chan string)
startTime := time.Now()
getAllText(h, 0, texts)
unbufSum += time.Since(startTime).Nanoseconds()
} else {
// Use buffering
bufCount += 1
texts := make(chan string, bufSize)
startTime := time.Now()
getAllText(h, bufSize, texts)
bufSum += time.Since(startTime).Nanoseconds()
}
}
unbufAvg := unbufSum / int64(unbufCount)
bufAvg := bufSum / int64(bufCount)
fmt.Printf("Unbuffered average time (ns): %v\n", unbufAvg)
fmt.Printf("Buffered average time (ns): %v\n", bufAvg)
}
// Kill the program and report the error
func die(err error) {
fmt.Printf("Terminating: %v\n", err.Error())
os.Exit(1)
}
// Walk through all of a nodes children and construct a string consisting
// of c.Data where c.Type == TextNode
func getAllText(n *gnhtml.Node, bufSize uint, texts chan string) string {
wg := sync.WaitGroup{}
// Go walk through all n's child nodes, sending only textual data
// over the texts channel.
wg.Add(1)
nTree := h5.NewTree(n)
go func () {
nTree.Walk(func (c *gnhtml.Node) {
if c.Type == gnhtml.TextNode {
texts <- c.Data
}
})
close(texts)
wg.Done()
}()
// As text data comes in over the texts channel, build up finalString
wg.Add(1)
finalString := ""
go func () {
for t := range texts {
finalString += t
}
wg.Done()
}()
// Return finalString once both of the goroutines have finished.
wg.Wait()
return finalString
}
With this (move channel creation before starting the timer) I have this result:
Buffered average time (ns): 2649
Buffered average time (ns): 2655
Buffered average time (ns): 2657
Buffered average time (ns): 2695
Buffered average time (ns): 2695
Buffered average time (ns): 2699
Buffered average time (ns): 2719
Buffered average time (ns): 2724
Buffered average time (ns): 2793
Buffered average time (ns): 3101
Unbuffered average time (ns): 2997
Unbuffered average time (ns): 3005
Unbuffered average time (ns): 3027
Unbuffered average time (ns): 3031
Unbuffered average time (ns): 3048
Unbuffered average time (ns): 3062
Unbuffered average time (ns): 3086
Unbuffered average time (ns): 3092
Unbuffered average time (ns): 3095
Unbuffered average time (ns): 3191
For improve, the solution are may be to keep the channel open when it's possible.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论