英文:
Thread-safe operation with Stdout and Stderr (exec. Cmd)
问题
我有一段代码,它能正常工作,但不具备线程安全性。在这些代码行中会发生竞争条件。我已经按照以下方式进行了重写:
readout, _ := cmd.StdoutPipe()
readerr, _ := cmd.StderrPipe()
链接:
https://play.golang.org/p/htbn2zXXeQk
我不喜欢这里使用了MultiReader
,而且我无法将stdout和stderr的数据分开。
r, _ := bufio.NewReader(io.MultiReader(readout, readerr)).ReadString('\n')
另外,第二个示例不起作用(在代码中已注释掉)。我期望stdout不为空(就像这里的示例:https://play.golang.org/p/8EY3i1Uk_aO)。
如何使逻辑与第一个示例相同,但又具备线程安全性?
英文:
I have code and it works in the correct way, but it isn’t thread-safety https://play.golang.org/p/8EY3i1Uk_aO in these rows race happens here.
stdout := cmd.Stdout.(*bytes.Buffer).String()
stderr := cmd.Stderr.(*bytes.Buffer).String()
I rewrote it in this way
readout, _ := cmd.StdoutPipe()
readerr, _ := cmd.StderrPipe()
The link
https://play.golang.org/p/htbn2zXXeQk
I don’t like that MultiReader
is used here and I cannot separate data stdout from stderr
r, _ := bufio.NewReader(io.MultiReader(readout, readerr)).ReadString('\n')
Also the second example doesn’t work (it is commented in the code). I expected stdout not to be empty (like here https://play.golang.org/p/8EY3i1Uk_aO)
How to make the logic like in the first example, but it should be thread-safety?
答案1
得分: 3
你必须在单独的goroutine中持续读取cmd.Stdout
和cmd.Stderr
,直到它们关闭。例如,就像你对cmd.Stdin
所做的那样(当然是在另一个方向上)。否则,存在死锁的风险-进程被阻塞在等待写入stdout/stderr,而你的程序被阻塞在等待进程完成。
或者,就像@JimB所说的那样,只需将字符串缓冲区分配给cmd.Stdout
和cmd.Stderr
,它们将在进程运行时填充。
func invoke(cmd *exec.Cmd) (stdout string, stderr string, err error) {
stdoutbuf, stderrbuf := new(strings.Builder), new(strings.Builder)
cmd.Stdout = stdoutbuf
cmd.Stderr = stderrbuf
err = cmd.Start()
if err != nil {
return
}
err = cmd.Wait()
return stdoutbuf.String(), stderrbuf.String(), err
}
演示链接:
https://play.golang.org/p/hakSVNbqirB
英文:
You have to pump cmd.Stdout
and cmd.Stderr
in separate goroutines until they are closed, for example, like you did with cmd.Stdin
(but in other direction, of course). Otherwise there's a risk of deadlock - the process is blocked waiting to write to stdout/stderr, and your program is blocked waiting for the process to finish.
Or, like @JimB said, just assign string buffers to cmd.Stdout
and cmd.Stderr
, they will be filled as the process runs.
func invoke(cmd *exec.Cmd) (stdout string, stderr string, err error) {
stdoutbuf, stderrbuf := new(strings.Builder), new(strings.Builder)
cmd.Stdout = stdoutbuf
cmd.Stderr = stderrbuf
err = cmd.Start()
if err != nil {
return
}
err = cmd.Wait()
return stdoutbuf.String(), stderrbuf.String(), err
}
Live demo:
答案2
得分: 0
我使用了@rusty的建议,无论如何,race line runner.go:264
是这样的:
append(normalizeEncoding(stdoutbuf.String()), normalizeEncoding(stderrbuf.String()), false)
英文:
I used the advice of @rusty, anyway the race
line `runner.go:264 ' is
append(normalizeEncoding(stdoutbuf.String()), normalizeEncoding(stderrbuf.String()), false)
答案3
得分: 0
解决方案:
创建自己的Writer包装器
type lockedWriter struct {
sync.RWMutex
buf []byte
w io.Writer
}
func (w *lockedWriter) Write(b []byte) (n int, err error) {
w.Lock()
defer w.Unlock()
w.buf = append(w.buf, b...)
return w.w.Write(b)
}
func (w *lockedWriter) String() string {
w.RLock()
defer w.RUnlock()
return string(w.buf)
}
用法
stdoutbuf, stderrbuf := &lockedWriter{w: new(strings.Builder)}, &lockedWriter{w: new(strings.Builder)}
cmd.Stdout = stdoutbuf
cmd.Stderr = stderrbuf
英文:
solution:
create your own Writer wrapper
type lockedWriter struct {
sync.RWMutex
buf []byte
w io.Writer
}
func (w *lockedWriter) Write(b []byte) (n int, err error) {
w.Lock()
defer w.Unlock()
w.buf = append(w.buf, b...)
return w.w.Write(b)
}
func (w *lockedWriter) String() string {
w.RLock()
defer w.RUnlock()
return string(w.buf)
}
usage
stdoutbuf, stderrbuf := &lockedWriter{w: new(strings.Builder)}, &lockedWriter{w: new(strings.Builder)}
cmd.Stdout = stdoutbuf
cmd.Stderr = stderrbuf
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论