Go – 使用goroutines执行Bash命令n次,并存储和打印其结果

huangapple go评论82阅读模式
英文:

Go - Execute a Bash Command n Times using goroutines and Store & Print its result

问题

我对Golang还不太熟悉,我正在尝试执行一个带有参数的bash命令,然后将输出存储在一个变量中并进行打印

我可以执行一次,或者使用循环来实现,就像下面的代码一样:

package main

import (
	"fmt"
	"os/exec"
	"os"
	"sync"
)


func main() {

	//默认输出
	var (
		cmdOut []byte
		err    error
	)

	//Bash命令
	cmd  := "./myCmd"
	//要传递给命令的参数
	args := []string{"arg1", "arg2", "arg3"}

	//执行命令
	if cmdOut, err = exec.Command(cmd, args...).Output(); err != nil {
		fmt.Fprintln(os.Stderr, "There was an error running "+cmd+" "+args[0]+args[1]+args[2], err)
		os.Exit(1)
	}
	//存储结果
	sha := string(cmdOut)
	//打印结果
	fmt.Println(sha)
}

这个代码可以正常工作,我可以轻松地读取输出

现在,我想要重复这个操作n次,使用goroutines

我尝试按照回答How would you define a pool of goroutines to be executed at once in Golang?的方式进行,但是我无法使其工作。

我尝试了以下代码:

package main

import (
	"fmt"
	"os/exec"
	"sync"
)


func main() {

	//Bash命令
	cmd  := "./myCmd"
	//要传递给命令的参数
	args := []string{"arg1", "arg2", "arg3"}

	//用于goroutines的共享通道
	tasks := make(chan *exec.Cmd, 64)

	//生成4个goroutines
	var wg sync.WaitGroup
	for i := 0; i < 4; i++ {
		wg.Add(1)
		go func() {
			for cmd := range tasks {
				cmd.Run()
			}
			wg.Done()
		}()
	}

	//生成任务
	for i := 0; i < 10; i++ {
		tasks <- exec.Command(cmd, args...)
		//在这里我应该以某种方式打印后一个命令的结果
	}
	close(tasks)

	//等待goroutines完成
	wg.Wait()

	fmt.Println("Done")

}

但是,我真的不知道如何存储并打印执行命令的i-result

我该如何实现这个?

提前感谢您的帮助,如果对问题有任何疑问,请留下评论。

英文:

I'm pretty new to Golang in general and I'm trying to execute a bash command with its arguments n times, then, store the Output in a variable and Print it.

I'm able to do it just one time, or just using loops like the following:

package main

import (
    &quot;fmt&quot;
    &quot;os/exec&quot;
    &quot;os&quot;
    &quot;sync&quot;
)


func main() {

    //Default Output
    var (
	    cmdOut []byte
	    err    error
    )

    //Bash Command
    cmd  := &quot;./myCmd&quot;
   //Arguments to get passed to the command
   args := []string{&quot;arg1&quot;, &quot;arg2&quot;, &quot;arg3&quot;}

    //Execute the Command
    if cmdOut, err = exec.Command(cmd, args...).Output(); err != nil {
	    fmt.Fprintln(os.Stderr, &quot;There was an error running &quot;+cmd+&quot; &quot;+args[0]+args[1]+args[2], err)
	    os.Exit(1)
    }
    //Store it
    sha := string(cmdOut)
    //Print it
    fmt.Println(sha)
}

This works just fine, I'm able to read the output easily.

Now, I would like to repeat this very same operation for n times, using goroutines.

I tried following the very same approach of the guy who answered How would you define a pool of goroutines to be executed at once in Golang? but I'm not able to make it work.

That's what I tried so far:

package main

import (
	&quot;fmt&quot;
	&quot;os/exec&quot;
	&quot;sync&quot;
)


func main() {

	//Bash Command
	cmd  := &quot;./myCmd&quot;
	//Arguments to get passed to the command
	 args := []string{&quot;arg1&quot;, &quot;arg2&quot;, &quot;arg3&quot;}

	//Common Channel for the goroutines
	tasks := make(chan *exec.Cmd, 64)

	//Spawning 4 goroutines
	var wg sync.WaitGroup
	for i := 0; i &lt; 4; i++ {
		wg.Add(1)
		go func() {
			for cmd := range tasks {
				cmd.Run()
			}
			wg.Done()
		}()
	}

	//Generate Tasks
	for i := 0; i &lt; 10; i++ {
		tasks &lt;- exec.Command(cmd, args...)
        //Here I should somehow print the result of the latter command
	}
	close(tasks)

	// wait for the workers to finish
	wg.Wait()

	fmt.Println(&quot;Done&quot;)

}

But, I don't really find out how to store the i-result of an executed command and print it.

How can I achieve this?

Thanks in advance, for any clarification on the question just leave a comment.

答案1

得分: 8

以下是翻译好的内容:

所以以下是修复你的问题的方法:

  • 你可以调用Cmd.Output()来获取命令的输出。否则,你可以将Cmd.StdOutPipe管道连接到一个byte.Buffer中,然后从中读取。

  • 你的goroutine逻辑是错误的。它只会执行4次,因为waitgroup会调用Done(),然后主函数会退出。你正确地在主函数中关闭了通道,以通知工作线程退出for循环。在此之后应该调用wg.Done(),所以我使用了defer来延迟调用。

  • 当你使用go命令执行匿名函数时,尽量不要从父作用域中捕获任何东西。这可能会导致灾难。相反,找出你需要的参数,并将它们传递给函数。

package main

import (
    "fmt"
    "os/exec"
    "sync"
)

func main() {
    cmd := "./foo.sh"
    // 要传递给命令的参数
    args := []string{"bar", "baz"}

    // 用于goroutine的共享通道
    tasks := make(chan *exec.Cmd, 64)

    // 启动4个goroutine
    var wg sync.WaitGroup
    for i := 0; i < 4; i++ {
        wg.Add(1)
        go func(num int, w *sync.WaitGroup) {
            defer w.Done()
            var (
                out []byte
                err error
            )
            for cmd := range tasks { // 当通道关闭时,循环将退出
                out, err = cmd.Output()
                if err != nil {
                    fmt.Printf("无法获取标准输出:%v", err)
                }
                fmt.Printf("goroutine %d 的命令输出:%s", num, string(out))
            }
        }(i, &wg)
    }

    // 生成任务
    for i := 0; i < 10; i++ {
        tasks <- exec.Command(cmd, args...)
    }
    close(tasks)

    // 等待工作线程完成
    wg.Wait()

    fmt.Println("完成")
}

希望对你有帮助!

英文:

so the following fixes your problem

  • you can call Cmd.Output() to get the output of the command.
    otherwise you could connect the Cmd.StdOutPipe pipe to a byte.Buffer for
    example and read from that.

  • your goroutine logic was wrong. it would only execute 4 times cause then the waitgroup would be Done() and main would exit. You are correct to close the channel from main to signal the workers to exit the for range loop. wg.Done should be called after that so i defered it.

  • when you execute anonymous functions with the go command try not to capture anything from the parent scope. it can lead to disaster. instead figure out which parameters you want and pass them to the function.

`

package main 

import (
    &quot;fmt&quot;
    &quot;os/exec&quot;
    &quot;sync&quot;
)
func main() {
    cmd := &quot;./foo.sh&quot;
    //Arguments to get passed to the command
    args := []string{&quot;bar&quot;, &quot;baz&quot;}

    //Common Channel for the goroutines
    tasks := make(chan *exec.Cmd, 64)

    //Spawning 4 goroutines
    var wg sync.WaitGroup
    for i := 0; i &lt; 4; i++ {
            wg.Add(1)
            go func(num int, w *sync.WaitGroup) {
                    defer w.Done()
                    var (
                            out []byte
                            err error
                    )
                    for cmd := range tasks { // this will exit the loop when the channel closes
                            out, err = cmd.Output()
                            if err != nil {
                                    fmt.Printf(&quot;can&#39;t get stdout:&quot;, err)
                            }
                            fmt.Printf(&quot;goroutine %d command output:%s&quot;, num, string(out))
                    }
            }(i, &amp;wg)
    }
    //Generate Tasks
    for i := 0; i &lt; 10; i++ {
            tasks &lt;- exec.Command(cmd, args...)
    }
    close(tasks)

    // wait for the workers to finish
    wg.Wait()

    fmt.Println(&quot;Done&quot;)

}

`

huangapple
  • 本文由 发表于 2016年10月26日 03:02:00
  • 转载请务必保留本文链接:https://go.coder-hub.com/40247726.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定