如何为包含通道、文件遍历和 API 调用的 Golang 程序编写单元测试?

huangapple go评论89阅读模式
英文:

How to write unit test for a program that contains channels, filewalk and api call in golang?

问题

我的程序整体如下所示。

func main() {

	flag.Parse()

	if *token == "" {
		log.Fatal(Red + "please provide a client token => -token={$token}")
	}

	tokenSource := oauth2.StaticTokenSource(&oauth2.Token{AccessToken: *token})
	oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
	client := putio.NewClient(oauthClient)

	//paths := make(chan string)
	var wg = new(sync.WaitGroup)
	for i := 0; i < 50; i++ {
		wg.Add(1)
		go worker(paths, wg, client)
	}
	WalkFilePath()
	//if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
	//	if err != nil {
	//		return fmt.Errorf("Failed to walk directory: %T %w", err, err)
	//	}
	//	if !info.IsDir() {
	//		paths <- path
	//	}
	//	return nil
	//}); err != nil {
	//	panic(fmt.Errorf("failed Walk: %w", err))
	//}
	close(paths)
	wg.Wait()
}

// walks the file path and sends paths to channel
func WalkFilePath() {
	if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
		if err != nil {
			return fmt.Errorf("Failed to walk directory: %T %w", err, err)
		}
		if !info.IsDir() {
			paths <- path
		}
		return nil
	}); err != nil {
		panic(fmt.Errorf("failed Walk: %w", err))
	}
}

func worker(paths <-chan string, wg *sync.WaitGroup, client *putio.Client) {
	defer wg.Done()
	for path := range paths {
		f, err := os.Open(path)
		if err != nil {
			log.Printf(Red + "Failed to open file %v for reading" + Reset, f.Name())
		}
		upload, err := client.Files.Upload(context.TODO(), f, path, 0)
		if err != nil {
			log.Printf(Red + "Failed to upload file %v" + Reset, upload.File.Name)
		}
		log.Printf(Green+ "File %v has been uploaded succesfully" + Reset, upload.File.Name)
	}
}

我写了这段代码。这是我能做到的最简洁的写法,我被告知要为程序编写一个单元测试。我感到困惑。例如,考虑WalkFilePath函数。为了测试这个函数,我应该提供什么样的输入,以及期望得到什么样的结果?因为它涉及到channel通信,也就是goroutine。有没有办法清晰地为这个程序编写单元测试?或者我应该改变代码结构,这对我来说不是很好。顺便说一下,程序运行正常。

英文:

My program is as follows as a whole.

func main() {
flag.Parse()
if *token == &quot;&quot; {
log.Fatal(Red + &quot;please provide a client token =&gt; -token={$token}&quot;)
}
tokenSource := oauth2.StaticTokenSource(&amp;oauth2.Token{AccessToken: *token})
oauthClient := oauth2.NewClient(context.TODO(), tokenSource)
client := putio.NewClient(oauthClient)
//paths := make(chan string)
var wg = new(sync.WaitGroup)
for i := 0; i &lt; 50; i++ {
wg.Add(1)
go worker(paths, wg, client)
}
WalkFilePath()
//if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
//	if err != nil {
//		return fmt.Errorf(&quot;Failed to walk directory: %T %w&quot;, err, err)
//	}
//	if !info.IsDir() {
//		paths &lt;- path
//	}
//	return nil
//}); err != nil {
//	panic(fmt.Errorf(&quot;failed Walk: %w&quot;, err))
//}
close(paths)
wg.Wait()
}
// walks the file path and sends paths to channel
func WalkFilePath() {
if err := filepath.Walk(*rootpath, func(path string, info os.FileInfo, err error) error {
if err != nil {
return fmt.Errorf(&quot;Failed to walk directory: %T %w&quot;, err, err)
}
if !info.IsDir() {
paths &lt;- path
}
return nil
}); err != nil {
panic(fmt.Errorf(&quot;failed Walk: %w&quot;, err))
}
}
func worker(paths &lt;-chan string, wg *sync.WaitGroup, client *putio.Client) {
defer wg.Done()
for path := range paths {
f, err := os.Open(path)
if err != nil {
log.Printf(Red + &quot;Failed to open file %v for reading&quot; + Reset, f.Name())
}
upload, err := client.Files.Upload(context.TODO(), f, path, 0)
if err != nil {
log.Printf(Red + &quot;Failed to upload file %v&quot; + Reset, upload.File.Name)
}
log.Printf(Green+ &quot;File %v has been uploaded succesfully&quot; + Reset, upload.File.Name)
}
}

I did write the code. That's the cleanest I can do and I was told to write a unit test for the program. I'm confused. For example, considering the WalkFilePath function. What should I provide and what kind of result I should expect to test the function. Because it contains channel communication meaning goroutines. Is there any way to write unit tests for this program clearly? Or should I change the code structure which is not good in this case for me. Btw, the program runs properly.

答案1

得分: 1

像大多数事物一样,Go语言对于如何进行测试有着非常明确的意见。请确保阅读https://go.dev/doc/tutorial/add-a-test。

例如,考虑WalkFilePath函数。我应该提供什么参数,并期望得到什么样的结果来测试这个函数?

WalkFilePath函数的输入应该是pathsrootpath。你的WalkFilePath函数没有从任何地方获取pathsrootpath,所以这段代码在原样情况下无法编译通过(当然,测试将有助于捕捉到这些问题)。

对于WalkFilePath的测试可以按照以下步骤进行:

  1. 在你的项目下的testdata/目录中创建一个文件系统结构,这是一个专门用于测试数据的目录。创建子目录和文件。例如,目录结构可以是这样的:

    testdata/
    walktest/
    dir1/
    file1.txt
    dir2/
    file2.txt
    dir3/
    file3.txt
    
  2. 现在你可以定义你将从通道中获取的预期数据。

    expected_paths := []string{
    "testdata/walktest/dir1/file1.txt",
    "testdata/walktest/dir2/file2.txt",
    "testdata/walktest/dir3/file3.txt",
    }
    
  3. 现在你需要修改WalkFilePath函数,使其接受rootpathpaths作为参数。

    func WalkFilePath(rootdir string, paths chan<- string) {
    
  4. 现在你可以编写测试了。

    func TestWalkFilePath(t *testing.T) {
    paths := make(chan string)
    go WalkFilePath("testdata/walktest", paths)
    results := make([]string, 0)
    for path := range paths {
    results = append(results, path)
    }
    exp, res := strings.Join(expected_paths, ""), strings.Join(results, "")
    if exp != res {
    t.Errorf("Expected %s got %s", exp, res)
    }
    }
    

因为它涉及到通道通信,也就是goroutine。

在单元测试中使用通道和goroutine是完全正常和有效的。

英文:

Like most things, Go is very opinionated about how to test. Make sure to read https://go.dev/doc/tutorial/add-a-test

> For example, considering the WalkFilePath function. What should I provide and what kind of result I should expect to test the function.

The input to WalkFilePath should be paths and a rootpath. Your WalkFilePath doesn't get paths or rootpath from anywhere, so this code wouldn't compile as is (testing will help catch that stuff of course).

A test for WalkFilePath might be done something like this:

  1. Create a filesystem structure in your project under testdata/, a directory expressly set aside for data used for testing. Create subdirectories and files. For an example that might look like:

    testdata/
    walktest/
    dir1/
    file1.txt
    dir2/
    file2.txt
    dir3/
    file3.txt
    
  2. Now you can define the expected data you'll be getting out of your channel.

    expected_paths := []string{
    &quot;testdata/walktest/dir1/file1.txt&quot;,
    &quot;testdata/walktest/dir2/file2.txt&quot;,
    &quot;testdata/walktest/dir3/file3.txt&quot;
    }
    
  3. Now you need to change WalkFilePath to take arguments for rootpath and paths.

    func WalkFilePath(rootdir string, paths chan&lt;- string) {
    
  4. Now you're ready to write your test.

    func TestWalkFilePath(t *testing.T(
    paths := make(chan string)
    go WalkFilePath(&quot;testdata/walktest&quot;)
    results := make([]string,0)
    for path := range paths {
    results = append(results, path)
    }
    exp, res := strings.Join(expected_paths, &quot;&quot;), strings.Join(results, &quot;&quot;)
    if exp != res {
    t.Errorf(&quot;Expected %s got %s&quot;, exp, res)
    }
    }
    

> Because it contains channel communication meaning goroutines.

It's totally normal and valid to use channels and goroutines in unit tests.

huangapple
  • 本文由 发表于 2021年12月5日 20:20:45
  • 转载请务必保留本文链接:https://go.coder-hub.com/70234235.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定