英文:
Go: receive os.cmd stdout and stderr in order
问题
我需要执行go中的子命令,并分别处理其stdout和stderr,同时保持输入/输出的顺序。我尝试了几种不同的方法,但无法实现正确的输出顺序;以下代码显示输出处理顺序是完全随机的:
package main
import (
"fmt"
"log"
"os/exec"
)
var (
result = ""
)
type writer struct {
result string
write func(bytes []byte)
}
func (writer *writer) Write(bytes []byte) (int, error) {
writer.result += string(bytes) // 后续处理结果
result += string(bytes)
return len(bytes), nil
}
func main() {
cmd := exec.Command("bash", "-c", "echo TEST1; echo TEST2 1>&2; echo TEST3")
stderr := &writer{}
cmd.Stderr = stderr
stdout := &writer{}
cmd.Stdout = stdout
err := cmd.Start()
if err != nil {
log.Fatal(err)
}
err = cmd.Wait()
if err != nil {
log.Fatal(err)
}
fmt.Println(result)
}
通过多次运行,代码可能会输出以下内容:
$ go run main.go
TEST1
TEST3
TEST2
我期望在所有情况下得到以下结果:
$ go run main.go
TEST1
TEST2
TEST3
我不能调用cmd.CombinedOutput
,因为我需要分别实时处理stdout/stderr。
英文:
I need to execute subcommand from go and process it stdout and stderr separately, with keeping order of ouput that comes to stdin/stdout. I've tried several differents ways, but could not achieve the correct order of output; following code shows that ouput handling order is absolutely random:
package main
import (
"fmt"
"log"
"os/exec"
)
var (
result = ""
)
type writer struct {
result string
write func(bytes []byte)
}
func (writer *writer) Write(bytes []byte) (int, error) {
writer.result += string(bytes) // process result later
result += string(bytes)
return len(bytes), nil
}
func main() {
cmd := exec.Command("bash", "-c", "echo TEST1; echo TEST2 1>&2; echo TEST3")
stderr := &writer{}
cmd.Stderr = stderr
stdout := &writer{}
cmd.Stdout = stdout
err := cmd.Start()
if err != nil {
log.Fatal(err)
}
err = cmd.Wait()
if err != nil {
log.Fatal(err)
}
fmt.Println(result)
}
With several runs code can output following:
$ go run main.go
TEST1
TEST3
TEST2
I expect following result in all cases:
$ go run main.go
TEST1
TEST2
TEST3
I can not call cmd.CombinedOutput because I need to process stdout/stderr separately and in realtime.
答案1
得分: 2
你正在执行的命令中没有"order"(顺序)的概念。它们是并行的管道,因此它们在效果上与两个 goroutine 并发执行的方式相同。你可以通过使用通道或互斥锁将它们按接收顺序存储并标记它们的来源。为了使输出在你的示例中不是随机的,你需要添加一点延迟。我已经成功地在真实命令中使用了这种方法:
package main
import (
"fmt"
"log"
"os/exec"
"sync"
)
var (
result = ""
)
type write struct {
source string
data string
}
type writer struct {
source string
mu *sync.Mutex
writes *[]write
}
func (w *writer) Write(bytes []byte) (int, error) {
w.mu.Lock()
defer w.mu.Unlock()
*w.writes = append(*w.writes, write{
source: w.source,
data: string(bytes),
})
return len(bytes), nil
}
func main() {
cmd := exec.Command("bash", "-c", "echo TEST1; sleep .1; echo TEST2 1>&2; sleep .1; echo TEST3")
var mu sync.Mutex
var writes []write
cmd.Stderr = &writer{
source: "STDERR",
mu: &mu,
writes: &writes,
}
cmd.Stdout = &writer{
source: "STDOUT",
mu: &mu,
writes: &writes,
}
err := cmd.Start()
if err != nil {
log.Fatal(err)
}
err = cmd.Wait()
if err != nil {
log.Fatal(err)
}
fmt.Printf("%q\n", writes)
}
将会产生以下输出:
[{"STDOUT" "TEST1\n"} {"STDERR" "TEST2\n"} {"STDOUT" "TEST3\n"}]
英文:
There is no "order" with the command that you're executing. They're parallel pipes, and as such they're effectively concurrent in the same way that two goroutines are concurrent. You can certainly store them in the order you receive them and tag them with their source by making use of channels or a mutex. In order to make the output not random with your synthetic example, you need to add a bit of a pause. I have used this method successfully with real commands, however:
package main
import (
"fmt"
"log"
"os/exec"
"sync"
)
var (
result = ""
)
type write struct {
source string
data string
}
type writer struct {
source string
mu *sync.Mutex
writes *[]write
}
func (w *writer) Write(bytes []byte) (int, error) {
w.mu.Lock()
defer w.mu.Unlock()
*w.writes = append(*w.writes, write{
source: w.source,
data: string(bytes),
})
return len(bytes), nil
}
func main() {
cmd := exec.Command("bash", "-c", "echo TEST1; sleep .1; echo TEST2 1>&2; sleep .1; echo TEST3")
var mu sync.Mutex
var writes []write
cmd.Stderr = &writer{
source: "STDERR",
mu: &mu,
writes: &writes,
}
cmd.Stdout = &writer{
source: "STDOUT",
mu: &mu,
writes: &writes,
}
err := cmd.Start()
if err != nil {
log.Fatal(err)
}
err = cmd.Wait()
if err != nil {
log.Fatal(err)
}
fmt.Printf("%q\n", writes)
}
will produce
[{"STDOUT" "TEST1\n"} {"STDERR" "TEST2\n"} {"STDOUT" "TEST3\n"}]
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论