How to write the output of an executable binary into memory rather than disk

I got a binary that works like the below:

> ./my_bin raw.avi output_file.avi

output_file.avi is what I want, some verbose information will print in the terminal when the job is succeeded, like:
> Copyright 2022 Company Inc... Success.

I want to run this command inside my code and redirect the output_file.avi into some byte array so that I don't have to read it from disk and delete it. My approach looks like the below Golang snippet:

func wrongOne(stdin []byte) ([]byte, error) {
	inBuf := bytes.NewBuffer(stdin)
	outBuf := bytes.NewBuffer(nil)
	cmd := exec.Command("./my_bin", "/dev/stdin", "/dev/stdout")
	cmd.Stdin = inBuf
	cmd.Stdout = outBuf
	err := cmd.Run()
	if err != nil {
		return nil, err
	}
	return outBuf.Bytes(), nil // wrong
}

However, the return byte array is longer than the below approach, which leads to failure on the MD5 check.

func correctOne(stdin []byte) ([]byte, error) {
	inBuf := bytes.NewBuffer(stdin)
	cmd := exec.Command("./my_bin", "/dev/stdin", "output_file")
	cmd.Stdin = inBuf
	err := cmd.Run()
	if err != nil {
		return nil, err
	}
	return os.ReadFile("output_file")
}

the wrongOne function can be modified to following code to be correct:

func modifiedWrongOne(stdin []byte) ([]byte, error) {
	inBuf := bytes.NewBuffer(stdin)
	outBuf := bytes.NewBuffer(nil)
	cmd := exec.Command("./my_bin", "/dev/stdin", "/dev/stdout")
	cmd.Stdin = inBuf
	cmd.Stdout = outBuf
	err := cmd.Run()
	if err != nil {
		return nil, err
	}
	correct, _ := correctOne(stdin)
	return outBuf.Bytes()[:len(correct)], nil // diff
}

I presume that the output verbose information is included in the /dev/stdout so that the wrongOne function doesn't works. i.e.,

> the output of wrongOne = the output of correctOne + []byte{"Copyright 2022 Company Inc... Success."}

Is there any solution that I can get the output_file.avi in the pipe without save it as file and read it from disk? Thanks!

The command writes the copyright notice to stdout. To avoid commingling the copyright notice with the output file, use a file other than /dev/stdout as the output file.

The function below uses Cmd.ExtraFiles to connect a pipe to fd 3 in the child process. The function copies data from the pipe to a byte buffer and returns those bytes to the caller.

func otherOne(stdin []byte) ([]byte, error) {
	r, w, err := os.Pipe()
	if err != nil {
		return nil, err
	}
	defer r.Close()
	defer w.Close()

	cmd := exec.Command("./my_bin", "/dev/stdin", "/dev/fd/3")
	cmd.Stdin = bytes.NewReader(stdin)
	cmd.ExtraFiles = []*os.File{w} // The first file is fd 3.
    if err := cmd.Start(); err != nil {
		return nil, err
	}
	w.Close()
	var outbuf bytes.Buffer
	if _, err := io.Copy(&outbuf, r); err != nil {
		return nil, err
	}
	if err := cmd.Wait(); err != nil {
		return nil, err
	}
	return outbuf.Bytes(), nil
}

After months, I figure out another way to solve this problem. The basic idea is similar with Cerise, i.e., using /dev/fd/3 to redirect the output file. After that, we redirect /dev/fd/3 to /dev/stdout, verbose log to /dev/stderr
by 3>&1, 1>&2, respectively. An additional gen.sh is added. Here's the solution:

#gen.sh
./mybin /dev/stdin /dev/fd/3 3>&1 1>&2

// gen.go
func gen(stdin []byte) ([]byte, error) {
	inBuf := bytes.NewBuffer(stdin)
	outBuf := bytes.NewBuffer(nil)
	cmd := exec.Command("./gen.sh")
	cmd.Stdin = inBuf
	cmd.Stdout = outBuf
	cmd.Stderr = os.Stderr
	err := cmd.Run()
	if err != nil {
		return nil, err
	}
	return outBuf.Bytes(), nil
}