使用exec.Cmd执行命令时,如何实现线程安全的标准输出和标准错误输出?

huangapple go评论65阅读模式
英文:

Thread-safe operation with Stdout and Stderr (exec. Cmd)

问题

我有一段代码,它能正常工作,但不具备线程安全性。在这些代码行中会发生竞争条件。我已经按照以下方式进行了重写:

readout, _ := cmd.StdoutPipe()
readerr, _ := cmd.StderrPipe()

链接:
https://play.golang.org/p/htbn2zXXeQk

我不喜欢这里使用了MultiReader,而且我无法将stdout和stderr的数据分开。

r, _ := bufio.NewReader(io.MultiReader(readout, readerr)).ReadString('\n')

另外,第二个示例不起作用(在代码中已注释掉)。我期望stdout不为空(就像这里的示例:https://play.golang.org/p/8EY3i1Uk_aO)。

如何使逻辑与第一个示例相同,但又具备线程安全性?

英文:

I have code and it works in the correct way, but it isn’t thread-safety https://play.golang.org/p/8EY3i1Uk_aO in these rows race happens here.

stdout := cmd.Stdout.(*bytes.Buffer).String()
stderr := cmd.Stderr.(*bytes.Buffer).String()

I rewrote it in this way

readout, _ := cmd.StdoutPipe()
readerr, _ := cmd.StderrPipe()

The link
https://play.golang.org/p/htbn2zXXeQk

I don’t like that MultiReader is used here and I cannot separate data stdout from stderr

r, _ := bufio.NewReader(io.MultiReader(readout, readerr)).ReadString('\n')

Also the second example doesn’t work (it is commented in the code). I expected stdout not to be empty (like here https://play.golang.org/p/8EY3i1Uk_aO)

How to make the logic like in the first example, but it should be thread-safety?

答案1

得分: 3

你必须在单独的goroutine中持续读取cmd.Stdoutcmd.Stderr,直到它们关闭。例如,就像你对cmd.Stdin所做的那样(当然是在另一个方向上)。否则,存在死锁的风险-进程被阻塞在等待写入stdout/stderr,而你的程序被阻塞在等待进程完成。

或者,就像@JimB所说的那样,只需将字符串缓冲区分配给cmd.Stdoutcmd.Stderr,它们将在进程运行时填充。

func invoke(cmd *exec.Cmd) (stdout string, stderr string, err error) {
    stdoutbuf, stderrbuf := new(strings.Builder), new(strings.Builder)
    cmd.Stdout = stdoutbuf
    cmd.Stderr = stderrbuf
    err = cmd.Start()
    if err != nil {
        return
    }
    err = cmd.Wait()
    return stdoutbuf.String(), stderrbuf.String(), err
}

演示链接:

https://play.golang.org/p/hakSVNbqirB

英文:

You have to pump cmd.Stdout and cmd.Stderr in separate goroutines until they are closed, for example, like you did with cmd.Stdin (but in other direction, of course). Otherwise there's a risk of deadlock - the process is blocked waiting to write to stdout/stderr, and your program is blocked waiting for the process to finish.

Or, like @JimB said, just assign string buffers to cmd.Stdout and cmd.Stderr, they will be filled as the process runs.

func invoke(cmd *exec.Cmd) (stdout string, stderr string, err error) {
	stdoutbuf, stderrbuf := new(strings.Builder), new(strings.Builder)
	cmd.Stdout = stdoutbuf
	cmd.Stderr = stderrbuf
	err = cmd.Start()
	if err != nil {
		return
	}
	err = cmd.Wait()
	return stdoutbuf.String(), stderrbuf.String(), err
}

Live demo:

https://play.golang.org/p/hakSVNbqirB

答案2

得分: 0

我使用了@rusty的建议,无论如何,race line runner.go:264 是这样的:

append(normalizeEncoding(stdoutbuf.String()), normalizeEncoding(stderrbuf.String()), false)

使用exec.Cmd执行命令时,如何实现线程安全的标准输出和标准错误输出?

英文:

I used the advice of @rusty, anyway the race
line `runner.go:264 ' is

append(normalizeEncoding(stdoutbuf.String()), normalizeEncoding(stderrbuf.String()), false)

使用exec.Cmd执行命令时,如何实现线程安全的标准输出和标准错误输出?

答案3

得分: 0

解决方案:
创建自己的Writer包装器

type lockedWriter struct {
	sync.RWMutex

	buf []byte
	w   io.Writer
}


func (w *lockedWriter) Write(b []byte) (n int, err error) {
	w.Lock()
	defer w.Unlock()

	w.buf = append(w.buf, b...)
	return w.w.Write(b)
}

func (w *lockedWriter) String() string {
	w.RLock()
	defer w.RUnlock()

	return string(w.buf)
}

用法

stdoutbuf, stderrbuf := &lockedWriter{w: new(strings.Builder)}, &lockedWriter{w: new(strings.Builder)}
cmd.Stdout = stdoutbuf
cmd.Stderr = stderrbuf
英文:

solution:
create your own Writer wrapper

type lockedWriter struct {
	sync.RWMutex

	buf []byte
	w   io.Writer
}


func (w *lockedWriter) Write(b []byte) (n int, err error) {
	w.Lock()
	defer w.Unlock()

	w.buf = append(w.buf, b...)
	return w.w.Write(b)
}

func (w *lockedWriter) String() string {
	w.RLock()
	defer w.RUnlock()

	return string(w.buf)
}

usage

stdoutbuf, stderrbuf := &lockedWriter{w: new(strings.Builder)}, &lockedWriter{w: new(strings.Builder)}
cmd.Stdout = stdoutbuf
cmd.Stderr = stderrbuf

huangapple
  • 本文由 发表于 2021年6月24日 21:11:51
  • 转载请务必保留本文链接:https://go.coder-hub.com/68116457.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定