使用流处理进行异步测试

huangapple
go评论121阅读模式
英文:

Asynchronous Testing With Stream Processing

问题

我对Go语言还很陌生，所以可能对Go的异步/流处理有一些基础的误解，但我会尽力回答你的问题。

你想使用ginkgo编写一些测试，测试一个处理流的函数。

处理函数从一个文件中按行读取文本，直到遇到特殊的分隔符行，然后尝试将文本解析为JSON。代码如下：

  1. func ParseConfig(inStream *os.File) (Config, error){
  2.   var header string
  3.   
  4.   var stdin = bufio.NewScanner(inStream)
  5.   for stdin.Scan() {
  6.     line := stdin.Text()
  7.     
  8.     if line == "|||" {
  9.       break;
  10.     }
  11.     
  12.     header += line
  13.   }
  14.   
  15.   // 在这里解析JSON并返回
  16. }

你的测试代码大致如下：

  1. Describe("ParseConfig()", func() {
  2.   It("should pass for a valid header", func(){
  3.     _, err := io.WriteString(stream, "{\"Key\": \"key\", \"File\": \"file\"}\n|||\n")
  4.     Expect(err).NotTo(HaveOccurred())
  6.     conf, err := parser.ParseConfig(stream)
  7.     Expect(err).NotTo(HaveOccurred())
  8.     
  9.     Expect(conf.Key).To(Equal("key"))
  10.   })
  11. })

不幸的是，这会导致JSON解析错误，因为它尝试解析一个空字符串。我猜问题可能是在告诉ParseConfig()函数监听数据之前，我就已经将字符串发送到流中了。但我不太清楚如何重构代码以使用正确的Go协程先监听数据再发送数据。

我看到一些潜在的解决方案涉及使用"通道"（我对此不太熟悉），但我担心这个需求可能不值得进行重大重构来引入全新的并发范例。

谢谢！

英文:

I'm very new to Go, so I may be misunderstanding something foundational about Go's async/stream handling, but here goes...

I'm trying to write some tests using ginkgo on a function I wrote that processes streams.

The processing side reads in newline-delimited text from a File until it encounters a special delimiter line at which point it tries to parse the text as JSON. The code looks like this:

  1. func ParseConfig(inStream *os.File) (Config, error){
  2.   var header string
  3.   
  4.   var stdin = bufio.NewScanner(inStream)
  5.   for stdin.Scan() {
  6.     line := stdin.Text()
  7.     
  8.     if line == "|||" {
  9.       break;
  10.     }
  11.     
  12.     header += line
  13.   }
  14.   
  15.   // parse JSON here and return
  16. }

My test looks something like this

  1. Describe("ParseConfig()", func() {
  2.   It("should pass for a valid header", func(){
  3.     _, err := io.WriteString(stream, "{\"Key\": \"key\", \"File\": \"file\"}\n|||\n")
  4.     Expect(err).NotTo(HaveOccurred())
  6.     conf, err := parser.ParseConfig(stream)
  7.     Expect(err).NotTo(HaveOccurred())
  8.     
  9.     Expect(conf.Key).To(Equal("key"))
  10.   })
  11. })

Unfortunately, this yields a JSON parsing error, as it's trying to parse an empty string. I'm assuming that my problem is that I'm sending the string on the stream before I've told the ParseConfig() function to listen on that string for data? But I'm not entirely clear how I could refactor this to use proper go routines to first listen for data then send it.

Some of the potential solutions I saw were around the use of "channels" (with which I'm unfamiliar) but I was worried that this one need might not be worth a major refactor to introduce a whole new paradigm of concurrency.

Thanks!

答案1

得分: 2

不确定我是否理解正确，但是你的ParseConfig函数应该接受一个io.Reader而不是*os.File。这样你就可以直接测试它，而不用担心并发问题。

t_test.go文件：

  1. package main
  3. import (
  4. 	"strings"
  5. 	"testing"
  7. 	"github.com/onsi/ginkgo"
  8. 	"github.com/onsi/gomega"
  9. )
  11. var _ = ginkgo.Describe("ParseConfig()", func() {
  12. 	ginkgo.It("should pass for a valid header", func() {
  13. 		// 真的不知道你在处理'stream'变量时在做什么
  14. 		// 这是一个测试，你应该构造一个测试场景并将其传递给你的配置函数
  15. 		stream := strings.NewReader(`{"Key": "key", "File": "file"}` + "\n|||\n")
  17. 		conf, err := ParseConfig(stream)
  18. 		gomega.Expect(err).NotTo(gomega.HaveOccurred())
  19. 		gomega.Expect(conf.Key).To(gomega.Equal("key"))
  20. 	})
  21. })
  23. func TestParseConfig(t *testing.T) {
  24. 	ginkgo.RunSpecs(t, "Parse Config")
  25. }

main.go文件：

  1. package main
  3. import (
  4. 	"bufio"
  5. 	"encoding/json"
  6. 	"io"
  7. 	"log"
  8. 	"os"
  9. )
  11. type Config struct {
  12. 	Key  string
  13. 	File string
  14. }
  16. func ParseConfig(inStream io.Reader) (*Config, error) {
  17. 	var header string
  19. 	var stdin = bufio.NewScanner(inStream)
  20. 	for stdin.Scan() {
  21. 		line := stdin.Text()
  23. 		if line == "|||" {
  24. 			break
  25. 		}
  26. 		header += line
  27. 	}
  29. 	c := &Config{}
  31. 	// 在这里解析JSON并返回
  32. 	if err := json.Unmarshal([]byte(header), c); err != nil {
  33. 		return nil, err
  34. 	}
  35. 	return c, nil
  36. }
  38. func main() {
  39. 	f, err := os.Open("config.json")
  40. 	if err != nil {
  41. 		log.Fatal(err)
  42. 	}
  43. 	ParseConfig(f)
  44. }
英文:

Not sure if I understood correctly, but your ParseConfig should probably take an io.Reader instead of a *os.File. That way you can test it directly without worrying about concurrency.

file t_test.go:

  1. package main
  3. import (
  4.         "strings"
  5.         "testing"
  7.         "github.com/onsi/ginkgo"
  8.         "github.com/onsi/gomega"
  9. )
  11. var _ = ginkgo.Describe("ParseConfig()", func() {
  12.         ginkgo.It("should pass for a valid header", func() {
  13.                 // really don't know what you were doing with your 'stream' variable
  14.                 // This is a test, you should forge a test scenario and pass it to your config function
  15.                 stream := strings.NewReader(`{"Key": "key", "File": "file"}` + "\n|||\n")
  17.                 conf, err := ParseConfig(stream)
  18.                 gomega.Expect(err).NotTo(gomega.HaveOccurred())
  19.                 gomega.Expect(conf.Key).To(gomega.Equal("key"))
  20.         })
  21. })
  23. func TestParseConfig(t *testing.T) {
  24.         ginkgo.RunSpecs(t, "Parse Config")
  25. }

file main.go

  1. package main
  3. import (
  4.         "bufio"
  5.         "encoding/json"
  6.         "io"
  7.         "log"
  8.         "os"
  9. )
  11. type Config struct {
  12.         Key  string
  13.         File string
  14. }
  16. func ParseConfig(inStream io.Reader) (*Config, error) {
  17.         var header string
  19.         var stdin = bufio.NewScanner(inStream)
  20.         for stdin.Scan() {
  21.                 line := stdin.Text()
  23.                 if line == "|||" {
  24.                         break
  25.                 }
  26.                 header += line
  27.         }
  29.         c := &Config{}
  31.         // parse JSON here and return
  32.         if err := json.Unmarshal([]byte(header), c); err != nil {
  33.                 return nil, err
  34.         }
  35.         return c, nil
  36. }
  38. func main() {
  39.         f, err := os.Open("config.json")
  40.         if err != nil {
  41.                 log.Fatal(err)
  42.         }
  43.         ParseConfig(f)
  44. }

huangapple
  • 本文由 发表于 2014年6月25日 03:56:24
  • 转载请务必保留本文链接：https://go.coder-hub.com/24395191.html
  • asynchronous
  • ginkgo
  • go
  • stream
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定