英文:
How to get live view of stdout and stderr of process which was created in c++ program?
问题
I want to run process inside c++ program and be able to capture it's stdout, stderr during the process's lifetime (I figured out the stdin part). For that, I am using boost.process (1.81.0) on ubuntu 22.04 (but I want the solution to be cross-platform). Ultimately, I want to build my custom ssh (just for fun), so I need to be able to control shell's stdout and stderr. I launch the test_program inside process_control and I want be able to see live stdout and stderr output, but it is captured only after test_program terminates, which happens when I feed end as an input. Here are the code samples of mentioned programs:
process_control.cpp
#include <boost/process.hpp>
#include <boost/process/pipe.hpp>
#include <boost/asio/io_service.hpp>
#include <thread>
#include <iostream>
int main() {
using namespace boost;
std::string output{};
std::string error{};
asio::io_service ios;
std::vector<char> vOut(128 << 10);
auto outBuffer{asio::buffer(vOut)};
process::async_pipe pipeOut(ios);
std::function<void(const system::error_code &ec, std::size_t n)> onStdOut;
onStdOut = [&](const system::error_code &ec, size_t n) {
std::cout << "onSTDOUT CALLED.\n";
output.reserve(output.size() + n);
output.insert(output.end(), vOut.begin(), vOut.begin() + static_cast<long>(n));
if (!ec) {
asio::async_read(pipeOut, outBuffer, onStdOut);
} else {
std::cout << "STDOUT ERROR\n";
}
std::cout << output << "\n";
};
std::vector<char> vErr(128 << 10);
auto errBuffer{asio::buffer(vErr)};
process::async_pipe pipeErr(ios);
std::function<void(const system::error_code &ec, std::size_t n)> onStdErr;
onStdErr = [&](const system::error_code &ec, size_t n) {
std::cout << "onSTDERR CALLED.\n";
error.reserve(error.size() + n);
error.insert(error.end(), vErr.begin(), vErr.begin() + static_cast<long>(n));
if (!ec) {
asio::async_read(pipeErr, errBuffer, onStdErr);
} else {
std::cout << "STDERR ERROR\n";
}
std::cout << error << "\n";
};
process::opstream in;
process::child c(
"test_program",
process::std_out > pipeOut,
process::std_err > pipeErr,
process::std_in < in,
ios
);
asio::async_read(pipeOut, outBuffer, onStdOut);
asio::async_read(pipeErr, errBuffer, onStdErr);
std::jthread t{[&ios] { ios.run(); }};
std::cout<<"STARTING LOOP: \n";
do {
std::string input_command{};
std::cout << "ENTER INPUT: ";
std::getline(std::cin, input_command);
if (c.running()) { //to prevent sigpipe if process dies during input
in << input_command << std::endl;
}
std::this_thread::yield();
} while (c.running());
return 0;
}
test_program.cpp
#include <iostream>
#include <chrono>
#include <thread>
using namespace std::chrono_literals;
int main() {
std::cout << "Started program.\n";
while(true){
std::cout << "Something\n";
std::cerr << "error stream\n";
std::this_thread::sleep_for(0.5s);
if(std::rand()%3==0){
std::cout << "Waiting for input...\n";
std::string input{};
std::getline(std::cin, input);
std::cout << "Got input: \"" << input << "\"\n";
if(input=="end"){
break;
}
}
}
return 0;
}
And the example output is: image
How to capture stdout and stderr during the process's (in this case test_program) life?
What am I doing wrong here?
I also want to merge stdout and stderr into one output and also keep the chronological order, but I guess that could be done with passing the same buffer.
I also tried redirecting streams in shell like this:
bash -c './test_program 2> stdout.txt 1> stderr.txt'
and it worked fine, but did not work when I tried the same in c++ code
process::child c(
"bash -c './test_program 2> stdout.txt 1> stderr.txt'",
process::std_in < in,
ios
);
and got output
STARTING LOOP:
ENTER INPUT: 2>: -c: line 1: unexpected EOF while looking for matching `''
2>: -c: line 2: syntax error: unexpected end of file
ls
end
or
std::vector<std::string> args{{"-c"},{"'./test_program 2> stdout.txt 1> stderr.txt'"}};
process::child c(
"bash", process::args(args),
process::std_in < in,
ios
);
and got output
terminate called after throwing an instance of 'boost::process::process_error'
what(): execve failed: No such file or directory
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
But redirecting to 2 separate files wouldn't really work for me, since I would like to maintain chronological order. So when tried bash -c './test_program 2> merged_output.txt 1> merged_output.txt' I was not surprised that the output wasn't looking good either.
EDIT:
I figured it out. For further reference, you can just simply create and use FILE instance, like this:
std::unique_ptr<FILE, decltype(&fclose)> p_stdout{fopen("output.txt", "w+"), fclose};
process::child c(
"test_program",
process::std_out > p_stdout.get(),
process::std_err > p_stdout.get(),
process::std_in < in,
ios
);
and then open the same file in read mode
std::unique_ptr<FILE, decltype(&fclose)> read_file{fopen("output.txt", "r"), fclose};
to read from it. You have to open it every time you want updated state, so I am not sure whether this approach is clean. But it works.
英文:
I want to run process inside c++ program and be able to capture it's stdout, stderr during the process's lifetime (I figured out the stdin part). For that, I am using boost.process (1.81.0) on ubuntu 22.04 (but I want the solution to be cross-platform). Ultimately, I want to build my custom ssh (just for fun), so I need to be able to control shell's stdout and stderr. I launch the test_program inside process_control and I want be able to see live stdout and stderr output, but it is captured only after test_program terminates, which happens when I feed end as an input. Here are the code samples of mentioned programs:
process_control.cpp
#include <boost/process.hpp>
#include <boost/process/pipe.hpp>
#include <boost/asio/io_service.hpp>
#include <thread>
#include <iostream>
int main() {
using namespace boost;
std::string output{};
std::string error{};
asio::io_service ios;
std::vector<char> vOut(128 << 10);
auto outBuffer{asio::buffer(vOut)};
process::async_pipe pipeOut(ios);
std::function<void(const system::error_code &ec, std::size_t n)> onStdOut;
onStdOut = [&](const system::error_code &ec, size_t n) {
std::cout << "onSTDOUT CALLED.\n";
output.reserve(output.size() + n);
output.insert(output.end(), vOut.begin(), vOut.begin() + static_cast<long>(n));
if (!ec) {
asio::async_read(pipeOut, outBuffer, onStdOut);
} else {
std::cout << "STDOUT ERROR\n";
}
std::cout << output << "\n";
};
std::vector<char> vErr(128 << 10);
auto errBuffer{asio::buffer(vErr)};
process::async_pipe pipeErr(ios);
std::function<void(const system::error_code &ec, std::size_t n)> onStdErr;
onStdErr = [&](const system::error_code &ec, size_t n) {
std::cout << "onSTDERR CALLED.\n";
error.reserve(error.size() + n);
error.insert(error.end(), vErr.begin(), vErr.begin() + static_cast<long>(n));
if (!ec) {
asio::async_read(pipeErr, errBuffer, onStdErr);
} else {
std::cout << "STDERR ERROR\n";
}
std::cout << error << "\n";
};
process::opstream in;
process::child c(
"test_program",
process::std_out > pipeOut,
process::std_err > pipeErr,
process::std_in < in,
ios
);
asio::async_read(pipeOut, outBuffer, onStdOut);
asio::async_read(pipeErr, errBuffer, onStdErr);
std::jthread t{[&ios] { ios.run(); }};
std::cout<<"STARTING LOOP: \n";
do {
std::string input_command{};
std::cout << "ENTER INPUT: ";
std::getline(std::cin, input_command);
if (c.running()) { //to prevent sigpipe if process dies during input
in << input_command << std::endl;
}
std::this_thread::yield();
} while (c.running());
return 0;
}
test_program.cpp
#include <iostream>
#include <chrono>
#include <thread>
using namespace std::chrono_literals;
int main() {
std::cout<<"Started program.\n";
while(true){
std::cout<<"Something\n";
std::cerr<<"error stream\n";
std::this_thread::sleep_for(0.5s);
if(std::rand()%3==0){
std::cout<<"Waiting for input...\n";
std::string input{};
std::getline(std::cin, input);
std::cout<<"Got input: \""<<input<<"\"\n";
if(input=="end"){
break;
}
}
}
return 0;
}
And the example output is:
image
How to capture stdout and stderr during the process's (in this case test_program) life?
What am I doing wrong here?
I also want to merge stdout and stderr into one output and also keep the chronological order, but I guess that could be done with passing the same buffer.
I also tried redirecting streams in shell like this:
bash -c './test_program 2> stdout.txt 1> stderr.txt'
and it worked fine, but did not work when I tried the same in c++ code
process::child c(
"bash -c './test_program 2> stdout.txt 1> stderr.txt'",
process::std_in < in,
ios
);
and got output
STARTING LOOP:
ENTER INPUT: 2>: -c: line 1: unexpected EOF while looking for matching `''
2>: -c: line 2: syntax error: unexpected end of file
ls
end
or
std::vector<std::string> args{{"-c"},{"'./test_program 2> stdout.txt 1> stderr.txt'"}};
process::child c(
"bash", process::args(args),
process::std_in < in,
ios
);
and got output
terminate called after throwing an instance of 'boost::process::process_error'
what(): execve failed: No such file or directory
Process finished with exit code 134 (interrupted by signal 6: SIGABRT)
But redirecting to 2 separate files wouldn't really work for me, since I would like to maintain chronological order. So when tried
bash -c './test_program 2> merged_output.txt 1> merged_output.txt'
I was not surprised that the output wasn't looking good either.
EDIT:
I figured it out. For further reference, you can just simply create and use FILE instance, like this:
std::unique_ptr<FILE, decltype(&fclose)> p_stdout{fopen("output.txt", "w+"), fclose};
process::child c(
"test_program",
process::std_out > p_stdout.get(),
process::std_err > p_stdout.get(),
process::std_in < in,
ios
);
and then open the same file in read mode
std::unique_ptr<FILE, decltype(&fclose)> read_file{fopen("output.txt", "r"), fclose};
to read from it. You have to open it every time you want updated state, so I am not sure whether this approach is clean. But it works.
答案1
得分: 1
我不确定你理解了什么。但我认为最重要的是 async_read 在管道关闭或缓冲区满时才会完成。考虑你的操作，这没有意义。
消除一些重复，我会使用 async_read_some：
#include <boost/asio/io_service.hpp>
#include <boost/process.hpp>
#include <boost/process/async.hpp>
#include <iomanip>
#include <iostream>
#include <thread>
namespace asio = boost::asio;
namespace bp = boost::process;
using namespace std::placeholders;
struct PipeReader {
using error_code = boost::system::error_code;
using stringbuf = asio::dynamic_string_buffer<char, std::char_traits<char>, std::allocator<char>>;
bp::async_pipe pipe;
std::string output;
stringbuf buf = asio::dynamic_buffer(output);
std::ostream& mirror;
PipeReader(asio::io_service& ios, std::ostream& stream) : pipe(ios), mirror(stream) {}
void read_loop(error_code ec = {}, size_t n = {}) {
std::cerr << "[fd:" << pipe.native_source() << " " << n << ", " << ec.message() << "]";
if (n) {
buf.commit(n);
if (mirror)
mirror << std::string_view(output).substr(output.size() - n) << std::flush;
}
if (!ec)
pipe.async_read_some(buf.prepare(512), std::bind(&PipeReader::read_loop, this, _1, _2));
}
};
int main() {
std::string error{};
asio::io_context ios;
PipeReader pipeOut(ios, std::cout), pipeErr(ios, std::cerr);
bp::opstream in;
bp::child c(
"./build/test_program",
bp::std_out > pipeOut.pipe,
bp::std_err > pipeErr.pipe,
bp::std_in < in,
ios);
pipeOut.read_loop();
pipeErr.read_loop();
std::jthread io_thread([&ios] { ios.run(); });
std::cout << "STARTING LOOP: \n";
for (std::string input_command{};
std::cout << "ENTER INPUT: " && std::getline(std::cin, input_command) && c.running();
std::this_thread::yield()) //
{
in << input_command << std::endl;
}
std::cout << "\n---\ntotal stdout:" << pipeOut.output.length() << ", stderr:" << pipeErr.output.length()
<< std::endl;
}
英文:
I'm not sure what you figured out. But I think the most important thing is that async_read doesn't complete unless the pipe is closed or the buffer is full. That makes no sense considering what you're doing.
Removing some duplication, I'd use async_read_some:
Live On Coliru (referring to essentially unmodified test_program)
#include <boost/asio/io_service.hpp>
#include <boost/process.hpp>
#include <boost/process/async.hpp>
#include <iomanip>
#include <iostream>
#include <thread>
namespace asio = boost::asio;
namespace bp = boost::process;
using namespace std::placeholders;
struct PipeReader {
using error_code = boost::system::error_code;
using stringbuf = asio::dynamic_string_buffer<char, std::char_traits<char>, std::allocator<char>>;
bp::async_pipe pipe;
std::string output;
stringbuf buf = asio::dynamic_buffer(output);
std::ostream& mirror;
PipeReader(asio::io_service& ios, std::ostream& stream) : pipe(ios), mirror(stream) {}
void read_loop(error_code ec = {}, size_t n = {}) {
std::cerr << "[fd:" << pipe.native_source() << " " << n << ", " << ec.message() << "]";
if (n) {
buf.commit(n);
if (mirror)
mirror << std::string_view(output).substr(output.size() - n) << std::flush;
}
if (!ec)
pipe.async_read_some(buf.prepare(512), std::bind(&PipeReader::read_loop, this, _1, _2));
}
};
int main() {
std::string error{};
asio::io_context ios;
PipeReader pipeOut(ios, std::cout), pipeErr(ios, std::cerr);
bp::opstream in;
bp::child c( //
"./build/test_program", //
bp::std_out > pipeOut.pipe, //
bp::std_err > pipeErr.pipe, //
bp::std_in < in, //
ios);
pipeOut.read_loop();
pipeErr.read_loop();
std::jthread io_thread([&ios] { ios.run(); });
std::cout << "STARTING LOOP: \n";
for (std::string input_command{};
std::cout << "ENTER INPUT: " && std::getline(std::cin, input_command) && c.running();
std::this_thread::yield()) //
{
in << input_command << std::endl;
}
std::cout << "\n---\ntotal stdout:" << pipeOut.output.length() << ", stderr:" << pipeErr.output.length()
<< std::endl;
}
Local side-by-side demo to avoid the jumbling that happens on Coliru:
On the right is the test_program directly, on the left using Boost Process child process. The left side in text:
[fd:6 0, Success][fd:8 0, Success]STARTING LOOP:
ENTER INPUT: [fd:6 27, Success]Started program.
Something
[fd:8 13, Success]error stream
[fd:6 10, Success]Something
[fd:8 13, Success]error stream
[fd:6 10, Success]Something
[fd:8 13, Success]error stream
[fd:6 21, Success]Waiting for input...
foor
ENTER INPUT: [fd:6 28, Success]Got input: "foor"
Something
[fd:8 13, Success]error stream
[fd:6 10, Success]Something
[fd:8 13, Success]error stream
[fd:6 10, Success]Something
[fd:8 13, Success]error stream
[fd:6 10, Success]Something
[fd:8 13, Success]error stream
[fd:6 10, Success]Something
[fd:8 13, Success]error stream
[fd:6 21, Success]Waiting for input...
bar
ENTER INPUT: [fd:6 27, Success]Got input: "bar"
Something
[fd:8 13, Success]error stream
[fd:6 21, Success]Waiting for input...
end
ENTER INPUT: [fd:6 17, Success]Got input: "end"
[fd:8 0, End of file][fd:6 0, End of file]
---
total stdout:734, stderr:629
<!-- -->
答案2
得分: 0
自从 test_program 从不输出 std::flush 或 std::endl（后者也执行刷新操作），当其输出被传输到管道时，将在内部进行缓冲，只有在调用 exit 时才会刷新。
如果您希望更早刷新行，请使用 std::endl 而不是 '\n'（这是它的用途），或在希望刷新的位置显式使用 std::flush。
英文:
Since test_program never outputs a std::flush or std::endl (which also does a flush), its output when going to a pipe will be buffered internally and will only be flushed when it calls exit.
If you want lines to be flushed sooner, use std::endl instead of '\n' (that's what it is for) or explicit std::flush at the points you want it to be flushed.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库，让每个人都能够通过互相帮助和分享经验来进步。
评论