Go exec.Command do not return output to stdout when I use cmd.Process.wait()

huangapple go评论84阅读模式
英文:

Go exec.Command do not return output to stdout when I use cmd.Process.wait()

问题

Go版本:
go version go1.18 linux/arm64

我的期望:
程序运行脚本,并在每次运行后返回输出结果。

发生了什么:
程序运行,有时不返回输出。

我故意使用了cmd.Process.wait(),因为下面的代码是我应用程序的一部分,需要运行脚本"service mysql start"。如果我使用cmd.Wait(),我的应用程序将等待mysql完成,这不是我想要的,所以我使用cmd.process.wait。

package main

import (
        "fmt"
        "bytes"
        "os/exec"
)

func ExecCommand(script string) ([]byte, error) {
        var buf bytes.Buffer
        cmd := exec.Command("sh", "-c", script)
        cmd.Stdout = &buf
        cmd.Stderr = &buf
        err := cmd.Start()
        if err != nil {
                return buf.Bytes(), err 
        }   
        _, err = cmd.Process.Wait()
        return buf.Bytes(), err 
}

func main() {
        start_num := 118261738305877
        for i:=0; i<50000; i++{
            buf, err := ExecCommand(fmt.Sprintf("echo 'Y = %d' | grep 'Y' | awk '{print $3}'", start_num))
            if err != nil {
                fmt.Println("cmd execute err at round", i)
            } else {
                if len(buf) == 0 { 
                    fmt.Println("cmd return nothing at round", i)
                } else {
                    if i % 100 == 0 { 
                        fmt.Println("cmd return ", string(buf), " at round ", i)
                    }   
                }   
            }   
        }   
}

这是我在Linux上的输出:

# go run test_go_process.go
cmd return  118261738305877
  at round  0
cmd return  118261738305877
  at round  100
cmd return nothing at round 115
cmd return  118261738305877
  at round  200
cmd return  118261738305877
  at round  300
cmd return  118261738305877
  at round  400
cmd return  118261738305877
  at round  2800
cmd return  118261738305877
  at round  2900
cmd return  118261738305877
  at round  3000
cmd return  118261738305877
  at round  3100
cmd return  118261738305877
  at round  3200
cmd return nothing at round 3290
cmd return  118261738305877
  at round  3300
cmd return  118261738305877

输出:cmd return nothing at round 115 表明在这次运行中,从ExecCommand函数中没有返回任何内容,我想知道为什么以及如何修复,除了编辑cmd.Process.wait()。非常感谢。

英文:

Go version:
go version go1.18 linux/arm64

What I expected:
The program run script and return output result in each run.

What happen:
The program run and sometimes does not return output.

I intentionally use cmd.Process.wait() because the code below is part of my application which needs to run script "service mysql start". If I used cmd.Wait(), my application would wait for mysql to finish and that is not what I want, so I use cmd.process.wait.

package main

import (
        &quot;fmt&quot;
        &quot;bytes&quot;
        &quot;os/exec&quot;
)

func ExecCommand(script string) ([]byte, error) {
        var buf bytes.Buffer
        cmd := exec.Command(&quot;sh&quot;, &quot;-c&quot;, script)
        cmd.Stdout = &amp;buf
        cmd.Stderr = &amp;buf
        err := cmd.Start()
        if err != nil {
                return buf.Bytes(), err 
        }   
        _, err = cmd.Process.Wait()
        return buf.Bytes(), err 
}

func main() {
        start_num := 118261738305877
        for i:=0; i&lt;50000; i++{
            buf, err := ExecCommand(fmt.Sprintf(&quot;echo &#39;Y = %d&#39; | grep &#39;Y&#39; | awk &#39;{print $3}&#39;&quot;, start_num))
            if err != nil {
                fmt.Println(&quot;cmd execute err at round&quot;, i)
            } else {
                if len(buf) == 0 { 
                    fmt.Println(&quot;cmd return nothing at round&quot;, i)
                } else {
                    if i % 100 == 0 { 
                        fmt.Println(&quot;cmd return &quot;, string(buf), &quot; at round &quot;, i)
                    }   
                }   
            }   
        }   
}

Here is my output on linux:

# go run test_go_process.go
cmd return  118261738305877
  at round  0
cmd return  118261738305877
  at round  100
cmd return nothing at round 115
cmd return  118261738305877
  at round  200
cmd return  118261738305877
  at round  300
cmd return  118261738305877
  at round  400
cmd return  118261738305877
  at round  2800
cmd return  118261738305877
  at round  2900
cmd return  118261738305877
  at round  3000
cmd return  118261738305877
  at round  3100
cmd return  118261738305877
  at round  3200
cmd return nothing at round 3290
cmd return  118261738305877
  at round  3300
cmd return  118261738305877

output:cmd return nothing at round 115 shows that at this run, nothing return from func ExecCommand and I am wondering why and how to fix that except edit cmd.Process.wait(). Thanks a lot in advance.

答案1

得分: 2

> 需要运行脚本 "service mysql start" [...] 如果我使用 cmd.Wait(),我的应用程序将等待 mysql 完成,这不是我想要的,所以我使用 cmd.process.wait。

这不准确,也是导致你的行为的原因。

首先,"service mysql start" 进程只运行到 mysql 启动,而不是运行到 mysql 完成。否则,当你在命令行中运行它时,它永远不会返回。

其次,os.Process.Wait 和 exec.Command.Wait 都会等待进程完成。实际上,exec.Command.Wait 调用了 os.Process.Wait。

区别在于 os.Process.Wait:

> Wait 等待进程退出,然后返回描述其状态和错误(如果有)的 ProcessState。

而 exec.Command.Wait(重点在于添加的部分):

> Wait 等待命令退出,并等待任何从 stdin 复制或从 stdout 或 stderr 复制的操作完成。

在你的程序中,你省略了必要的同步,以确保你的进程输出被完全消耗。没有这个同步,有时可能没有完全消耗,有时可能会。

> output:cmd return nothing at round 115 显示在这次运行中,从函数 ExecCommand 没有返回任何内容。

如果我在 playground 中运行你的代码,结果是不同的,每次都会丢失 stdout 数据。(可能是因为你的环境比 go playground 环境中的 CPU 并发更多)。

但是如果我改为使用 cmd.Wait(),每次都可以正常工作。

Go playground 不允许你调用 50,000 个子 shell 进程,但足够让你看到行为上的差异。

英文:

> needs to run script "service mysql start" [ ... ] If I used cmd.Wait(), my application would wait for mysql to finish and that is not what I want, so I use cmd.process.wait.

That's not accurate, and it's the thing causing your behavior.

First of all, the service mysql start process does not run until mysql finishes, only until it starts. Otherwise it would never return when you ran it at the command line.

Second, os.Process.Wait and exec.Command.Wait both do wait for the process to complete. In fact, exec.Command.Wait calls os.Process.Wait.

The difference is that os.Process.Wait

> Wait waits for the Process to exit, and then returns a ProcessState describing its status and an error, if any.

Whereas exec.Command.Wait (emphasis added)

> Wait waits for the command to exit and waits for any copying to stdin or copying from stdout or stderr to complete.

In your program, you're omitting the synchronization required to make sure that your process's output is fully consumed. Without this synchronization, sometimes it may not be, and sometimes it may be.

> output:cmd return nothing at round 115 shows that at this run, nothing return from func ExecCommand

If I run your code in the playground, the results are different, with missing stdout data in every case. (Probably because your environment has more available CPU concurrency than the go playground environment).

But if I change to cmd.Wait(), it works every time.

The Go playground isn't willing to let you invoke 50,000 subshell processes, but it's enough that you see the difference in behavior.

huangapple
  • 本文由 发表于 2022年3月19日 17:50:09
  • 转载请务必保留本文链接:https://go.coder-hub.com/71537104.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定