Is there a concept of reading the output of a program line by line, as a stream?

I have a shell command (e.g. journalctl -f -o json) that continuously streams lines to the standard output.

I would like to retrieve this output line by line and process it further.

The documentation of os/exec addresses how to read the output of a command, and io deals with stream buffering.

Everywhere I looked, the handling goes through a fixed buffer that is read into, handled, and written further. My problem is that the size of this buffer is fixed and independent of the content.

Is there a way to read an incoming stream (in my case - the output of a shell command) line by line? Possibly with a library more high-level than io readers?

Use Cmd.StdoutPipe() to obtain the (piped) output of the process before you start it with Cmd.Start() (Start() starts the command but does not wait for it to complete).

And use a bufio.Scanner to read an input (io.Reader) line-by-line.

For example I'm gonna use this bash script that prints the current time 3 times, sleeping 1 second between them:

for i in {1..3}; do date; sleep 1; done

Example executing this and reading its output line-by-line:

cmd := exec.Command("bash", "-c", "for i in {1..3}; do date; sleep 1; done")
out, err := cmd.StdoutPipe()
if err != nil {
	log.Fatal(err)
}
defer out.Close()

err = cmd.Start()
if err != nil {
	log.Fatal(err)
}

scanner := bufio.NewScanner(out)
for scanner.Scan() {
	line := scanner.Text()
	fmt.Println("Output:", line)
}

Example output:

2022/11/29 14:38:48 Output: Tue Nov 29 02:38:48 PM CET 2022
2022/11/29 14:38:49 Output: Tue Nov 29 02:38:49 PM CET 2022
2022/11/29 14:38:50 Output: Tue Nov 29 02:38:50 PM CET 2022

(The first date-time at the beginning of each line is from the log package, to verify each line is printed after a second delay, the other timestamp is the output of the date command.)