如何在Golang中并发地通过STDIN/STDOUT连接多个程序进行读写？

huangapple
go评论111阅读模式
英文:

How to wire up multiple programs reading/writing via STDIN/STDOUT in Golang concurrently?

问题

在高层次上，我想要实现以下内容。每个方框都是一个从标准输入读取并写入标准输出的运行程序。我想编写一个 Go 语言程序来设置并运行这些程序，以便所有的生产和消费都是并行进行的。我考虑使用 io.Pipe、通道和 os.Exec 等方法。

  1.                             +-----------+                                  
  2.                             |  PROG-1   +-----------------------+          
  3.                 +---------> |           |                       v          
  4.                 |           +-----------+                                  
  5.                 |                                           +-------+      
  6.     +-----------+                                           | DIFF  +----->
  7.     | GENERATOR |                                           |       |      
  8.     +-----------+                                           +---+---+      
  9.                 |                                               ^          
  10.                 |                                               |          
  11.                 |           +-----------+                       |          
  12.                 |           |           |                       |          
  13.                 +---------> |  PROG-2   +-----------------------+          
  14.                             +-----------+

这是一个尝试，但似乎不太可靠，而且 "DIFF" 部分没有实现。

  1. package main
  3. import (
  4. 	"io"
  5. 	"os"
  6. 	"os/exec"
  7. )
  9. const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
  10. const LOWER = "abcdefghijklmnopqrstuvwxyz"
  12. func runProg(r io.Reader, cmd *exec.Cmd) {
  13. 	cmd.Stdin = r
  14. 	cmd.Stdout = os.Stdout // 我希望这部分输出传递给一个名为 "diff" 的第三个程序调用。
  15. 	cmd.Run()
  16. }
  18. func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {
  19. 	r1, w1 := io.Pipe()
  20. 	r2, w2 := io.Pipe()
  22. 	go runProg(r1, prog1)
  23. 	go runProg(r2, prog2)
  25. 	go func() {
  26. 		defer w1.Close()
  27. 		defer w2.Close()
  28. 		mw := io.MultiWriter(w1, w2)
  29. 		io.Copy(mw, r)
  30. 	}()
  32. }
  34. func main() {
  35. 	generator := exec.Command("ls", "-l")
  36. 	r, w := io.Pipe()
  37. 	generator.Stdout = w
  39. 	prog1 := exec.Command("tr", LOWER, UPPER)
  40. 	prog2 := exec.Command("tr", UPPER, LOWER)
  42. 	runIt(r, prog1, prog2)
  44. 	generator.Run()
  46. }
英文:

At a high level I would like to accomplish the following. Each box is a running program reading from STDIN and writing to STDOUT. I want to write a golang program which sets this up and runs it so that all production/consumption is happening in parallel. I am thinking of using io.Pipe, channels, and os.Exec etc.

  1.                             +-----------+                                  
  2.                             |  PROG-1   +-----------------------+          
  3.                 +---------> |           |                       v          
  4.                 |           +-----------+                                  
  5.                 |                                           +-------+      
  6.     +-----------+                                           | DIFF  +----->
  7.     | GENERATOR |                                           |       |      
  8.     +-----------+                                           +---+---+      
  9.                 |                                               ^          
  10.                 |                                               |          
  11.                 |           +-----------+                       |          
  12.                 |           |           |                       |          
  13.                 +---------> |  PROG-2   +-----------------------+          
  14.                             +-----------+

Here's an attempt but it doesn't seem to be working reliably and also the "DIFF" part is not implemented.

  1. package main
  3. import (
  4. 	"io"
  5. 	"os"
  6. 	"os/exec"
  7. )
  9. const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
  10. const LOWER = "abcdefghijklmnopqrstuvwxyz"
  12. func runProg(r io.Reader, cmd *exec.Cmd) {
  13. 	cmd.Stdin = r
  14. 	cmd.Stdout = os.Stdout // I want this to go to a third prog call "diff".
  15. 	cmd.Run()
  16. }
  18. func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {
  19. 	r1, w1 := io.Pipe()
  20. 	r2, w2 := io.Pipe()
  22. 	go runProg(r1, prog1)
  23. 	go runProg(r2, prog2)
  25. 	go func() {
  26. 		defer w1.Close()
  27. 		defer w2.Close()
  28. 		mw := io.MultiWriter(w1, w2)
  29. 		io.Copy(mw, r)
  30. 	}()
  32. }
  34. func main() {
  35. 	generator := exec.Command("ls", "-l")
  36. 	r, w := io.Pipe()
  37. 	generator.Stdout = w
  39. 	prog1 := exec.Command("tr", LOWER, UPPER)
  40. 	prog2 := exec.Command("tr", UPPER, LOWER)
  42. 	runIt(r, prog1, prog2)
  44. 	generator.Run()
  46. }

答案1

得分: 3

这里有几个问题。创建所有这些管道会增加工作量和复杂性。此外，使用Cmd.Start()和Cmd.Wait()可以内置并发运行命令。

  1. package main
  3. import (
  4. 	"fmt"
  5. 	"io"
  6. 	"os"
  7. 	"os/exec"
  8. )
  10. const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
  11. const LOWER = "abcdefghijklmnopqrstuvwxyz"
  13. func runProg(cmd *exec.Cmd) (w io.WriteCloser, err error) {
  14. 	w, err = cmd.StdinPipe()
  15. 	if err != nil {
  16. 		fmt.Println(err)
  17. 	}
  18. 	cmd.Stdout = os.Stdout
  19. 	err = cmd.Start()
  20. 	return w, err
  21. }
  23. func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {
  25. 	w1, err := runProg(prog1)
  26. 	if err != nil {
  27. 		fmt.Println(err)
  28. 	}
  29. 	w2, err := runProg(prog2)
  30. 	if err != nil {
  31. 		fmt.Println(err)
  32. 	}
  34. 	go func() {
  35. 		defer w1.Close()
  36. 		defer w2.Close()
  37. 		mw := io.MultiWriter(w1, w2)
  38. 		io.Copy(mw, r)
  39. 	}()
  41. }
  43. func main() {
  44. 	generator := exec.Command("ls", "-l")
  45. 	r, err := generator.StdoutPipe()
  46. 	if err != nil {
  47. 		fmt.Println(err)
  48. 	}
  50. 	prog1 := exec.Command("tr", LOWER, UPPER)
  51. 	prog2 := exec.Command("tr", UPPER, LOWER)
  53. 	runIt(r, prog1, prog2)
  55. 	generator.Run()
  57. 	err = prog1.Wait()
  58. 	err1 := prog2.Wait()
  59. 	if err != nil || err1 != nil {
  60. 		fmt.Println(err, err1)
  61. 	}
  62. }
英文:

There are a couple things here. You're adding work and complexity in creating all those pipes. Also, running the command concurrently is built-in using Cmd.Start() and Cmd.Wait().

  1. package main
  2. import (
  3. "fmt"
  4. "io"
  5. "os"
  6. "os/exec"
  7. )
  8. const UPPER = "ABCDEFGHIJKLMNOPQRSTUVWXYZ"
  9. const LOWER = "abcdefghijklmnopqrstuvwxyz"
  10. func runProg(cmd *exec.Cmd) (w io.WriteCloser, err error) {
  11. w, err := cmd.StdinPipe()
  12. if err != nil {
  13. fmt.Println(err)
  14. }
  15. cmd.Stdout = os.Stdout
  16. err = cmd.Start()
  17. }
  18. func runIt(r io.Reader, prog1 *exec.Cmd, prog2 *exec.Cmd) {
  19. w1, err := runProg(prog1)
  20. if err != nil {
  21. fmt.Println(err)
  22. }
  23. w2, err := runProg(prog2)
  24. if err != nil {
  25. fmt.Println(err)
  26. }
  27. go func() {
  28. defer w1.Close()
  29. defer w2.Close()
  30. mw := io.MultiWriter(w1, w2)
  31. io.Copy(mw, r)
  32. }()
  33. }
  34. func main() {
  35. generator := exec.Command("ls", "-l")
  36. r, err := generator.StdoutPipe()
  37. if err != nil {
  38. fmt.Println(err)
  39. }
  40. prog1 := exec.Command("tr", LOWER, UPPER)
  41. prog2 := exec.Command("tr", UPPER, LOWER)
  42. runIt(r, prog1, prog2)
  43. generator.Run()
  44. err = prog1.Wait()
  45. err1 := prog2.Wait()
  46. if err != nil || err1 != nil {
  47. fmt.Println(err, err1)
  48. }
  49. }

huangapple
  • 本文由 发表于 2015年9月13日 08:23:47
  • 转载请务必保留本文链接：https://go.coder-hub.com/32545103.html
  • concurrency
  • go
  • pipe
  • producer-consumer
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定