英文:
Why is this program not performing better with goroutines?
问题
我正在学习Go编程语言。请考虑以下程序,
package main
import (
"fmt"
"bytes"
"os"
"os/exec"
"path/filepath"
"sync"
)
func grep(file string) {
defer wg.Done()
cmd := exec.Command("grep", "-H", "--color=always", "add", file)
var out bytes.Buffer
cmd.Stdout = &out
cmd.Run()
fmt.Printf("%s\n", out.String())
}
func walkFn(path string, info os.FileInfo, err error) error {
if !info.IsDir() {
wg.Add(1)
go grep (path)
}
return nil
}
var wg sync.WaitGroup
func main() {
filepath.Walk("/tmp/", walkFn)
wg.Wait()
}
该程序遍历/tmp
目录中的所有文件,并在每个文件上执行grep
命令。因此,这将生成n
个goroutine,其中n
是/tmp
目录中存在的文件数。主函数等待所有goroutine完成工作。
有趣的是,使用goroutine和不使用goroutine执行该程序所需的时间相同。尝试运行go grep (path, c)
和grep (path, c)
(在执行此操作时,需要注释掉通道相关的内容)。
我原本期望使用goroutine的版本会更快,因为多个grep同时运行。但实际上它们几乎以相等的时间执行。我想知道为什么会这样?
英文:
I am learning Go programming language. Please consider the following program,
package main
import (
"fmt"
"bytes"
"os"
"os/exec"
"path/filepath"
"sync"
)
func grep(file string) {
defer wg.Done()
cmd := exec.Command("grep", "-H", "--color=always", "add", file)
var out bytes.Buffer
cmd.Stdout = &out
cmd.Run()
fmt.Printf("%s\n", out.String())
}
func walkFn(path string, info os.FileInfo, err error) error {
if !info.IsDir() {
wg.Add(1)
go grep (path)
}
return nil
}
var wg sync.WaitGroup
func main() {
filepath.Walk("/tmp/", walkFn)
wg.Wait()
}
This program walks all the files in the /tmp
directory, and does a grep
on each file in a goroutine. So this will spawn n
goroutines where n
is the number of files present in the /tmp
directory. Main waits till all goroutines finishes the work.
Interestingly, this program take same time to execute with and without goroutines. Try running go grep (path, c)
and grep (path, c)
(you need to comment channel stuff when doing this).
I was expecting goroutine version to run faster as multiple grep runs concurrently. But it executes almost in equal time. I am wondering why this happens?
答案1
得分: 6
尝试使用更多的核心。此外,为了比较的目的,使用一个更好的根目录,比如Go目录。SSD也会有很大的差异。例如,
func main() {
runtime.GOMAXPROCS(runtime.NumCPU())
goroot := "/home/peter/go/"
filepath.Walk(goroot, walkFn)
wg.Wait()
fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))
}
GOMAXPROCS: 1
real 0m10.137s
user 0m2.628s
sys 0m6.472s
GOMAXPROCS: 4
real 0m3.284s
user 0m2.492s
sys 0m5.116s
英文:
Try using more cores. Also, use a better root directory for comparative purposes, like the Go directory. An SSD makes a big difference too. For example,
func main() {
runtime.GOMAXPROCS(runtime.NumCPU())
goroot := "/home/peter/go/"
filepath.Walk(goroot, walkFn)
wg.Wait()
fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))
}
GOMAXPROCS: 1
real 0m10.137s
user 0m2.628s
sys 0m6.472s
GOMAXPROCS: 4
real 0m3.284s
user 0m2.492s
sys 0m5.116s
答案2
得分: 4
你的程序的性能受到磁盘速度(或者如果/tmp是一个内存磁盘的话,受到内存速度)的限制:计算是I/O限制的。无论有多少个goroutine并行运行,它都不能比这个速度更快地读取。
英文:
Your program's performance is bound to the speed of the disk (or ram, if /tmp is a ram disk): the computation is I/O bound. No matter how many goroutines run in parallel it can't read faster than that.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论