为什么这个程序使用goroutines后表现没有变得更好?

huangapple go评论111阅读模式
英文:

Why is this program not performing better with goroutines?

问题

我正在学习Go编程语言。请考虑以下程序,

package main

import (
    "fmt"
    "bytes"
    "os"
    "os/exec"
    "path/filepath"
    "sync"
)

func grep(file string) {
    defer wg.Done()

    cmd := exec.Command("grep", "-H", "--color=always", "add", file)
    var out bytes.Buffer
    cmd.Stdout = &out
    cmd.Run()
    fmt.Printf("%s\n", out.String())
}

func walkFn(path string, info os.FileInfo, err error) error {
    if !info.IsDir() {
        wg.Add(1)
        go grep (path)
    }
    return nil
}

var wg sync.WaitGroup

func main() {
    filepath.Walk("/tmp/", walkFn)
    wg.Wait()
}

该程序遍历/tmp目录中的所有文件,并在每个文件上执行grep命令。因此,这将生成n个goroutine,其中n/tmp目录中存在的文件数。主函数等待所有goroutine完成工作。

有趣的是,使用goroutine和不使用goroutine执行该程序所需的时间相同。尝试运行go grep (path, c)grep (path, c)(在执行此操作时,需要注释掉通道相关的内容)。

我原本期望使用goroutine的版本会更快,因为多个grep同时运行。但实际上它们几乎以相等的时间执行。我想知道为什么会这样?

英文:

I am learning Go programming language. Please consider the following program,

package main

import (
    "fmt"
    "bytes"
    "os"
    "os/exec"
    "path/filepath"
    "sync"
)

func grep(file string) {
    defer wg.Done()

    cmd := exec.Command("grep", "-H", "--color=always", "add", file)
    var out bytes.Buffer
    cmd.Stdout = &out
    cmd.Run()
    fmt.Printf("%s\n", out.String())
}

func walkFn(path string, info os.FileInfo, err error) error {
    if !info.IsDir() {
        wg.Add(1)
        go grep (path)
    }
    return nil
}

var wg sync.WaitGroup

func main() {
    filepath.Walk("/tmp/", walkFn)
    wg.Wait()
}

This program walks all the files in the /tmp directory, and does a grep on each file in a goroutine. So this will spawn n goroutines where n is the number of files present in the /tmp directory. Main waits till all goroutines finishes the work.

Interestingly, this program take same time to execute with and without goroutines. Try running go grep (path, c) and grep (path, c) (you need to comment channel stuff when doing this).

I was expecting goroutine version to run faster as multiple grep runs concurrently. But it executes almost in equal time. I am wondering why this happens?

答案1

得分: 6

尝试使用更多的核心。此外,为了比较的目的,使用一个更好的根目录,比如Go目录。SSD也会有很大的差异。例如,

func main() {
    runtime.GOMAXPROCS(runtime.NumCPU())
    goroot := "/home/peter/go/"
    filepath.Walk(goroot, walkFn)
    wg.Wait()
    fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))
}

GOMAXPROCS: 1
real	0m10.137s
user	0m2.628s
sys     0m6.472s

GOMAXPROCS: 4
real	0m3.284s
user	0m2.492s
sys     0m5.116s
英文:

Try using more cores. Also, use a better root directory for comparative purposes, like the Go directory. An SSD makes a big difference too. For example,

func main() {
	runtime.GOMAXPROCS(runtime.NumCPU())
	goroot := "/home/peter/go/"
	filepath.Walk(goroot, walkFn)
	wg.Wait()
	fmt.Println("GOMAXPROCS:", runtime.GOMAXPROCS(0))
}

GOMAXPROCS: 1
real	0m10.137s
user	0m2.628s
sys     0m6.472s

GOMAXPROCS: 4
real	0m3.284s
user	0m2.492s
sys     0m5.116s

答案2

得分: 4

你的程序的性能受到磁盘速度(或者如果/tmp是一个内存磁盘的话,受到内存速度)的限制:计算是I/O限制的。无论有多少个goroutine并行运行,它都不能比这个速度更快地读取。

英文:

Your program's performance is bound to the speed of the disk (or ram, if /tmp is a ram disk): the computation is I/O bound. No matter how many goroutines run in parallel it can't read faster than that.

huangapple
  • 本文由 发表于 2013年6月27日 21:05:48
  • 转载请务必保留本文链接:https://go.coder-hub.com/17343984.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定