将跳过的测试添加到覆盖范围内

huangapple go评论86阅读模式
英文:

Add Skipped Tests to Coverage

问题

我创建了connect.goconnect_test.go。由于某种原因,我需要跳过connect_test.go中的一些测试(使用t.Skip()),特别是涉及到与外部API的联系并可能涉及一些支付的测试。

然而,尽管我非常确定它们是有效的,因为我已经测试过它们,并且此后没有进行任何更改,但这些测试在流水线中降低了我的代码覆盖率。

我正在使用以下命令:

go test -v -coverprofile=profile.cov -covermode=count ./...

有没有办法将这些测试标记为“可信任”,或以任何方式将它们添加到覆盖范围内?

英文:

I created say connect.go and connect_test.go. For some reason I need to skip some tests in connect_test.go (using t.Skip()), especially when it comes to contacting external APIs and may involve some payments.

However, those tests are decreasing my code coverage in the pipeline, although I am pretty sure that they're working for one simple reason: I already tested them, and didn't make any change thereafter.

I am using:

go test -v -coverprofile=profile.cov -covermode=count ./...

Is there any way to mark those tests as trusted or add them to the coverage by any means?

答案1

得分: 1

有没有办法将这些测试标记为可信任的或以任何方式将它们添加到覆盖范围内?

没有。

特别是因为不是(跳过或执行的)_测试_决定了行覆盖率百分比,而是由这些测试所执行的“实际”代码。因此,将测试标记为“可信任”将需要记住该测试执行的哪些实际代码行。

英文:

"Is there any way to mark those tests as trusted or add them to the coverage by any means?"

No.

Especially no as not the (skipped or executed) test determines line coverage percentage but "actual" code being exercised by these tests. So marking a test as "trusted" would have to remember which actual code lines would have been executed by this test.

答案2

得分: 1

覆盖率是在执行测试期间计算的,通过向你的(非测试)代码添加遥测来报告执行的代码。

如果你跳过一个测试,那么通常由该测试执行的代码将不会被执行,因此不会对覆盖率结果产生贡献。

唯一的方法是“信任”一个测试并使其对覆盖率产生贡献,就是构建一些机制,使得测试仍然运行,但所有测试都评估为“通过”,而不考虑实际结果。也就是说,运行测试但忽略任何错误。

这完全违背了这些测试的目的。

你可以使用go test包来实现这一点,通过创建interposer测试函数并使用它们来替代testing.T函数,例如:

var ignoreFailures bool

func TErrorf(t *testing T, s string, args ...any) {
  if ignoreFailures {
    return
  }
  t.Errorf(s, args...)
}

// then in a test...
func Test(t *testing.T) {
  ignoreFailures = true
  ...
  if wanted != got {
    TErrorf(t, "wanted: %s, got: %s", wanted, got)
  }

(对testing.T的所有函数重复)

但是不要这样做!

测试的整个目的是确保没有任何变化破坏了你的代码,以便你不依赖于“它在我的机器上工作”或“以前它工作过”。

如果你希望保留testing.T函数行为的其他方面,例如T.Fatal()(以避免在T.Fatal否则会阻止的测试中发生panic),则需要做更多的工作。

替代方法

我怀疑你跳过测试是因为它们由于外部依赖关系而难以执行或成本高昂。

interposer函数无法解决这个问题,而且很可能会使情况变得更糟,因为你仍然会尝试调用外部API,并可能遇到超时,从而导致测试运行更慢。

更好的解决方案是花时间模拟这些依赖关系,以便你可以独立于它们执行测试。

也就是说,不要通过实际调用外部API来测试你的代码,而是模拟API并创建测试来测试可能的情况。

结果,你的测试可能会更快地运行,并且可以在测试运行的环境中无论这些API是否可访问都可以运行。

如果你需要针对真实API执行集成测试,这些测试应该在一个单独的项目中,只有在需要时才能运行。

英文:

No. Coverage is calculated during the execution of tests by adding telemetry to your (non-test) code that reports which code is executed.

If you skip a test then the code that would normally be executed by that test is NOT executed, and so will not contribute to coverage results.

The only way to "trust" a test and have it still contribute to coverage would be to contrive some mechanism for the test to still run but have all tests evaluate as "passed", irrespective of the actual outcome. i.e. run the test but ignore any errors.

Which completely defeats the purpose of those tests.

You could do this using the go test package by creating interposer test functions and using those instead of the testing.T functions directly, e.g.

var ignoreFailures bool

func TErrorf(t *testing T, s string, args ...any) {
  if ignoreFailures {
    return
  }
  t.Errorf(s, args...)
}

// then in a test...
func Test(t *testing.T) {
  ignoreFailures = true
  ...
  if wanted != got {
    TErrorf(t, "wanted: %s, got: %s", wanted, got)
  }

(repeated for all functions of testing.T)

But do NOT do this!

The whole point of a test is to make absolutely certain that nothing has changed to break your code so that you aren't relying on "it works on my machine" or "it worked before".

And you will need to do more work if you wish to preserve other aspects of testing.T function behaviours, such as T.Fatal() (to avoid panics in your tests that T.Fatal otherwise would prevent).

Alternative Approach

I suspect you are skipping the tests because they are difficult/costly to execute due to external dependencies.

Interposer functions won't solve that and will likely make things worse as you will still be trying to call the external apis and possibly encountering timeouts, making your tests run even slower as a result.

A better solution is to take the time to mock those dependencies so that you can perform your tests independently of them.

i.e. instead of testing your code by actually calling external apis, mock the apis and create tests that exercise your code for the possible scenarios.

Your tests will likely run more quickly as a result and can run whether or not those apis are physically accessible from the environment in which the tests are running.

If you need integration tests to perform against the real apis, these should be in a separate project that can be run only if/when/where needed.

huangapple
  • 本文由 发表于 2023年6月14日 01:04:47
  • 转载请务必保留本文链接:https://go.coder-hub.com/76467219.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定