如何在Golang中并发地通过STDIN/STDOUT连接多个程序进行读写?

huangapple go评论85阅读模式
英文:

How to wire up multiple programs reading/writing via STDIN/STDOUT in Golang concurrently?

问题

在高层次上,我想要实现以下内容。每个方框都是一个从标准输入读取并写入标准输出的运行程序。我想编写一个 Go 语言程序来设置并运行这些程序,以便所有的生产和消费都是并行进行的。我考虑使用 io.Pipe、通道和 os.Exec 等方法。

                            +-----------+                                  
                            |  PROG-1   +-----------------------+          
                +---------> |           |                       v          
                |           +-----------+                                  
                |                                           +-------+      
    +-----------+                                           | DIFF  +----->
    | GENERATOR |                                           |       |      
    +-----------+                                           +---+---+      
                |                                               ^          
                |                                               |          
                |           +-----------+                       |          
                |           |           |                       |          
                +---------> |  PROG-2   +-----------------------+          
                            +-----------+                                  

这是一个尝试,但似乎不太可靠,而且 "DIFF" 部分没有实现。

package main

import (
	"io"
	"os"
	"os/exec"
)

const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
const LOWER = "abcdefghijklmnopqrstuvwxyz"

func runProg(r io.Reader, cmd *exec.Cmd) {
	cmd.Stdin = r
	cmd.Stdout = os.Stdout // 我希望这部分输出传递给一个名为 "diff" 的第三个程序调用。
	cmd.Run()
}

func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {
	r1, w1 := io.Pipe()
	r2, w2 := io.Pipe()

	go runProg(r1, prog1)
	go runProg(r2, prog2)

	go func() {
		defer w1.Close()
		defer w2.Close()
		mw := io.MultiWriter(w1, w2)
		io.Copy(mw, r)
	}()

}

func main() {
	generator := exec.Command("ls", "-l")
	r, w := io.Pipe()
	generator.Stdout = w

	prog1 := exec.Command("tr", LOWER, UPPER)
	prog2 := exec.Command("tr", UPPER, LOWER)

	runIt(r, prog1, prog2)

	generator.Run()

}
英文:

At a high level I would like to accomplish the following. Each box is a running program reading from STDIN and writing to STDOUT. I want to write a golang program which sets this up and runs it so that all production/consumption is happening in parallel. I am thinking of using io.Pipe, channels, and os.Exec etc.

                            +-----------+                                  
                            |  PROG-1   +-----------------------+          
                +---------> |           |                       v          
                |           +-----------+                                  
                |                                           +-------+      
    +-----------+                                           | DIFF  +----->
    | GENERATOR |                                           |       |      
    +-----------+                                           +---+---+      
                |                                               ^          
                |                                               |          
                |           +-----------+                       |          
                |           |           |                       |          
                +---------> |  PROG-2   +-----------------------+          
                            +-----------+                                  

Here's an attempt but it doesn't seem to be working reliably and also the "DIFF" part is not implemented.

package main

import (
	"io"
	"os"
	"os/exec"
)

const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
const LOWER = "abcdefghijklmnopqrstuvwxyz"

func runProg(r io.Reader, cmd *exec.Cmd) {
	cmd.Stdin = r
	cmd.Stdout = os.Stdout // I want this to go to a third prog call "diff".
	cmd.Run()
}

func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {
	r1, w1 := io.Pipe()
	r2, w2 := io.Pipe()

	go runProg(r1, prog1)
	go runProg(r2, prog2)

	go func() {
		defer w1.Close()
		defer w2.Close()
		mw := io.MultiWriter(w1, w2)
		io.Copy(mw, r)
	}()

}

func main() {
	generator := exec.Command("ls", "-l")
	r, w := io.Pipe()
	generator.Stdout = w

	prog1 := exec.Command("tr", LOWER, UPPER)
	prog2 := exec.Command("tr", UPPER, LOWER)

	runIt(r, prog1, prog2)

	generator.Run()

}

答案1

得分: 3

这里有几个问题。创建所有这些管道会增加工作量和复杂性。此外,使用Cmd.Start()和Cmd.Wait()可以内置并发运行命令。

package main

import (
	"fmt"
	"io"
	"os"
	"os/exec"
)

const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
const LOWER = "abcdefghijklmnopqrstuvwxyz"

func runProg(cmd *exec.Cmd) (w io.WriteCloser, err error) {
	w, err = cmd.StdinPipe()
	if err != nil {
		fmt.Println(err)
	}
	cmd.Stdout = os.Stdout
	err = cmd.Start()
	return w, err
}

func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {

	w1, err := runProg(prog1)
	if err != nil {
		fmt.Println(err)
	}
	w2, err := runProg(prog2)
	if err != nil {
		fmt.Println(err)
	}

	go func() {
		defer w1.Close()
		defer w2.Close()
		mw := io.MultiWriter(w1, w2)
		io.Copy(mw, r)
	}()

}

func main() {
	generator := exec.Command("ls", "-l")
	r, err := generator.StdoutPipe()
	if err != nil {
		fmt.Println(err)
	}

	prog1 := exec.Command("tr", LOWER, UPPER)
	prog2 := exec.Command("tr", UPPER, LOWER)

	runIt(r, prog1, prog2)

	generator.Run()

	err = prog1.Wait()
	err1 := prog2.Wait()
	if err != nil || err1 != nil {
		fmt.Println(err, err1)
	}
}
英文:

There are a couple things here. You're adding work and complexity in creating all those pipes. Also, running the command concurrently is built-in using Cmd.Start() and Cmd.Wait().

package main
import (
"fmt"
"io"
"os"
"os/exec"
)
const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
const LOWER = "abcdefghijklmnopqrstuvwxyz"
func runProg(cmd *exec.Cmd) (w io.WriteCloser, err error) {
w, err := cmd.StdinPipe()
if err != nil {
fmt.Println(err)
}
cmd.Stdout = os.Stdout
err = cmd.Start()
}
func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {
w1, err := runProg(prog1)
if err != nil {
fmt.Println(err)
}
w2, err := runProg(prog2)
if err != nil {
fmt.Println(err)
}
go func() {
defer w1.Close()
defer w2.Close()
mw := io.MultiWriter(w1, w2)
io.Copy(mw, r)
}()
}
func main() {
generator := exec.Command("ls", "-l")
r, err := generator.StdoutPipe()
if err != nil {
fmt.Println(err)
}
prog1 := exec.Command("tr", LOWER, UPPER)
prog2 := exec.Command("tr", UPPER, LOWER)
runIt(r, prog1, prog2)
generator.Run()
err = prog1.Wait()
err1 := prog2.Wait()
if err != nil || err1 != nil {
fmt.Println(err, err1)
}
}

huangapple
  • 本文由 发表于 2015年9月13日 08:23:47
  • 转载请务必保留本文链接:https://go.coder-hub.com/32545103.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定