Go:按顺序接收 os.cmd 的标准输出和标准错误输出。

huangapple go评论78阅读模式
英文:

Go: receive os.cmd stdout and stderr in order

问题

我需要执行go中的子命令,并分别处理其stdout和stderr,同时保持输入/输出的顺序。我尝试了几种不同的方法,但无法实现正确的输出顺序;以下代码显示输出处理顺序是完全随机的:

package main

import (
    "fmt"
    "log"
    "os/exec"
)

var (
    result = ""
)

type writer struct {
    result string
    write  func(bytes []byte)
}

func (writer *writer) Write(bytes []byte) (int, error) {
    writer.result += string(bytes) // 后续处理结果
    result += string(bytes)
    return len(bytes), nil
}

func main() {
    cmd := exec.Command("bash", "-c", "echo TEST1; echo TEST2 1>&2; echo TEST3")

    stderr := &writer{}
    cmd.Stderr = stderr

    stdout := &writer{}
    cmd.Stdout = stdout

    err := cmd.Start()
    if err != nil {
        log.Fatal(err)
    }

    err = cmd.Wait()
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(result)
}

通过多次运行,代码可能会输出以下内容:

$ go run main.go
TEST1
TEST3
TEST2

我期望在所有情况下得到以下结果:

$ go run main.go
TEST1
TEST2
TEST3

我不能调用cmd.CombinedOutput,因为我需要分别实时处理stdout/stderr。

英文:

I need to execute subcommand from go and process it stdout and stderr separately, with keeping order of ouput that comes to stdin/stdout. I've tried several differents ways, but could not achieve the correct order of output; following code shows that ouput handling order is absolutely random:

package main

import (
    "fmt"
    "log"
    "os/exec"
)

var (
    result = ""
)

type writer struct {
    result string
    write  func(bytes []byte)
}

func (writer *writer) Write(bytes []byte) (int, error) {
    writer.result += string(bytes) // process result later
    result += string(bytes)
    return len(bytes), nil
}

func main() {
    cmd := exec.Command("bash", "-c", "echo TEST1; echo TEST2 1>&2; echo TEST3")

    stderr := &writer{}
    cmd.Stderr = stderr

    stdout := &writer{}
    cmd.Stdout = stdout

    err := cmd.Start()
    if err != nil {
        log.Fatal(err)
    }

    err = cmd.Wait()
    if err != nil {
        log.Fatal(err)
    }

    fmt.Println(result)
}

With several runs code can output following:

$ go run main.go
TEST1
TEST3
TEST2

I expect following result in all cases:

$ go run main.go
TEST1
TEST2
TEST3

I can not call cmd.CombinedOutput because I need to process stdout/stderr separately and in realtime.

答案1

得分: 2

你正在执行的命令中没有"order"(顺序)的概念。它们是并行的管道,因此它们在效果上与两个 goroutine 并发执行的方式相同。你可以通过使用通道或互斥锁将它们按接收顺序存储并标记它们的来源。为了使输出在你的示例中不是随机的,你需要添加一点延迟。我已经成功地在真实命令中使用了这种方法:

package main

import (
    "fmt"
    "log"
    "os/exec"
    "sync"
)

var (
    result = ""
)

type write struct {
    source string
    data   string
}

type writer struct {
    source string

    mu     *sync.Mutex
    writes *[]write
}

func (w *writer) Write(bytes []byte) (int, error) {
    w.mu.Lock()
    defer w.mu.Unlock()
    *w.writes = append(*w.writes, write{
        source: w.source,
        data:   string(bytes),
    })
    return len(bytes), nil
}

func main() {
    cmd := exec.Command("bash", "-c", "echo TEST1; sleep .1; echo TEST2 1>&2; sleep .1; echo TEST3")

    var mu sync.Mutex
    var writes []write

    cmd.Stderr = &writer{
        source: "STDERR",
        mu:     &mu,
        writes: &writes,
    }
    cmd.Stdout = &writer{
        source: "STDOUT",
        mu:     &mu,
        writes: &writes,
    }

    err := cmd.Start()
    if err != nil {
        log.Fatal(err)
    }

    err = cmd.Wait()
    if err != nil {
        log.Fatal(err)
    }

    fmt.Printf("%q\n", writes)
}

将会产生以下输出:

[{"STDOUT" "TEST1\n"} {"STDERR" "TEST2\n"} {"STDOUT" "TEST3\n"}]
英文:

There is no "order" with the command that you're executing. They're parallel pipes, and as such they're effectively concurrent in the same way that two goroutines are concurrent. You can certainly store them in the order you receive them and tag them with their source by making use of channels or a mutex. In order to make the output not random with your synthetic example, you need to add a bit of a pause. I have used this method successfully with real commands, however:

package main
import (
"fmt"
"log"
"os/exec"
"sync"
)
var (
result = ""
)
type write struct {
source string
data   string
}
type writer struct {
source string
mu     *sync.Mutex
writes *[]write
}
func (w *writer) Write(bytes []byte) (int, error) {
w.mu.Lock()
defer w.mu.Unlock()
*w.writes = append(*w.writes, write{
source: w.source,
data:   string(bytes),
})
return len(bytes), nil
}
func main() {
cmd := exec.Command("bash", "-c", "echo TEST1; sleep .1; echo TEST2 1>&2; sleep .1; echo TEST3")
var mu sync.Mutex
var writes []write
cmd.Stderr = &writer{
source: "STDERR",
mu:     &mu,
writes: &writes,
}
cmd.Stdout = &writer{
source: "STDOUT",
mu:     &mu,
writes: &writes,
}
err := cmd.Start()
if err != nil {
log.Fatal(err)
}
err = cmd.Wait()
if err != nil {
log.Fatal(err)
}
fmt.Printf("%q\n", writes)
}

will produce

[{"STDOUT" "TEST1\n"} {"STDERR" "TEST2\n"} {"STDOUT" "TEST3\n"}]

huangapple
  • 本文由 发表于 2015年8月19日 17:57:54
  • 转载请务必保留本文链接:https://go.coder-hub.com/32092275.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定