英文:
Confusing behaviour when streaming exec command outputs from c programs and shell scripts
问题
我写了一个测试程序来分析今天导致错误的代码的行为，以更好地理解它的行为。结果恰恰相反。
这是测试程序。它应该执行一个测试命令并将命令的输出流传输到标准输出。
import (
"bufio"
"fmt"
"io"
"os/exec"
)
func main() {
cmd1 := exec.Command("./testcommands/testcommand.sh")
execCmd(cmd1)
cmd2 := exec.Command("./testcommands/testcommand")
execCmd(cmd2)
}
func execCmd(cmd *exec.Cmd) {
stderr, _ := cmd.StderrPipe()
stdout, _ := cmd.StdoutPipe()
multi := io.MultiReader(stdout, stderr)
scanner := bufio.NewScanner(multi)
cmd.Start()
for scanner.Scan() {
m := scanner.Text()
fmt.Println(m)
}
cmd.Wait()
}
这两个被调用的测试命令基本上是相同的。一个是用bash实现的。
#!/bin/bash
for i in `seq 1 10` ; do
echo "run $i"
sleep 1
done
另一个是用C语言实现的。
#include <stdio.h>
#include <unistd.h>
int main() {
int i;
for (i=1; i<=10; i++) {
printf("run %d\n", i);
sleep(1);
}
return 0;
}
Shell脚本的输出被实时传输（每秒一行），然而C程序的输出只有在程序完全结束后才到达（10秒后一次性输出10行）。
这超出了我的理解范围。我甚至不确定这是否按预期工作，我可能遗漏了什么，或者是否应该提交一个错误报告 - 如果是这样的话，我甚至不确定是针对bash、golang还是C，或者可能是我不知道的一些Linux问题。
英文:
I wrote a test program to analyse the behaviour of a piece of code that caused a bug today, to understand its behaviour better. The opposite happend.
This is the test program. It should execute a testcommand and stream the commands output to stdout.
import (
"bufio"
"fmt"
"io"
"os/exec"
)
func main() {
cmd1 := exec.Command("./testcommands/testcommand.sh")
execCmd(cmd1)
cmd2 := exec.Command("./testcommands/testcommand")
execCmd(cmd2)
}
func execCmd(cmd *exec.Cmd) {
stderr, _ := cmd.StderrPipe()
stdout, _ := cmd.StdoutPipe()
multi := io.MultiReader(stdout, stderr)
scanner := bufio.NewScanner(multi)
cmd.Start()
for scanner.Scan() {
m := scanner.Text()
fmt.Println(m)
}
cmd.Wait()
}
The two testcommands called do basically the same. One is implemented in bash
#!/bin/bash
for i in `seq 1 10` ; do
echo "run $i"
sleep 1
done
The other one in C
#include <stdio.h>
#include <unistd.h>
int main() {
int i;
for (i=1; i<=10; i++) {
printf("run %d\n", i);
sleep(1);
}
return 0;
}
The output of the shell script does get streamed (1 line per second), however the output of the c program only arrives after the program is finished completely (10 lines at once after 10 seconds).
This goes way over my head. I'm not even sure if this is working as intended and i'm just missing something, or if i should open a bug report - and if so, i'm not even sure if it'll be for bash, golang or c. Or maybe it's some linux thing i don't know about.
答案1
得分: 3
当stdout（printf写入的地方）连接到终端时，stdout将是行缓冲的，并且输出会在每个换行符时刷新（实际写入）。
但是当stdout没有连接到终端时，例如用于重定向或管道，它将是完全缓冲的。完全缓冲意味着输出只有在缓冲区变满（在你的小例子中不太可能）或显式刷新（例如使用fflush(stdout)）时才会被写入。
Go的exec功能可能会为程序的输入和输出创建管道。
为了解决你的问题，你的C程序需要在每次printf调用之后调用fflush：
printf("run %d\n", i);
fflush(stdout);
英文:
When stdout (to which printf writes) is connected to a terminal, then stdout will be line-buffered and output will be flushed (actually written) on each newline.
But when stdout is not connected to a terminal, like for example it's used for redirection or a pipe, then it's fully buffered. Fully buffered means the output only will be written when the buffer becomes full (unlikely in your small example) or when explicitly flushed (for example with fflush(stdout)).
What the Go exec functionality probably does is to create pipes for the programs input and output.
To solve your problem, your C program needs to call fflush after each printf call:
printf("run %d\n", i);
fflush(stdout);
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论