如何识别使用Jest覆盖率测试涵盖但实际未测试的代码行。

huangapple go评论69阅读模式
英文:

How to identify which lines are covered but not actually tested using Jest Coverage

问题

每当测试执行一行代码,它将被视为已覆盖。

但在你的测试中,可能会发生没有关于这行代码的断言的情况。

让我们来看一个非常简化的React组件:

function BasicComponent() {
  return <div>
    <h1>Hello there</h1>
    <p>This is a basic component</p>
  </div>;
}

现在,如果我使用JestReact-Testing-Library编写一个简单的测试:

import { render, screen } from '@testing-library/react';

describe('<BasicComponent />', () => {
  it('should render correctly', () => {
    render(<BasicComponent />);

    expect(screen.getByText('Hello there')).toBeInTheDocument();
  });
});

这个测试将渲染BasicComponent,执行其中的每一行代码。

但如果你仔细看测试,你会注意到我们从未断言文档中是否存在文本"This is a basic component"。

尽管未经测试,但代码覆盖率仍然为100%。

我想知道是否有一种方法可以找出哪些行代码被覆盖但实际上没有经过测试?

英文:

Whenever a line is executed by a test, it'll be considered as covered.

But in your test, it can happen that there is no assertion regarding this line.

Let's take a really simplified React component:

function BasicComponent() {
  return &lt;div&gt;
    &lt;h1&gt;Hello there&lt;/h1&gt;
    &lt;p&gt;This is a basic component&lt;/p&gt;
  &lt;/div&gt;;
}

Now, if I write a simple test using Jest and React-Testing-Library:

import { render, screen } from &#39;@testing-library/react&#39;;

describe(&#39;&lt;BasicComponent /&gt;&#39;, () =&gt; {
  it(&#39;should render correctly&#39;, () =&gt; {
    render(&lt;BasicComponent /&gt;);

    expect(screen.getByText(&#39;Hello there&#39;)).toBeInTheDocument();
  });
});

This test will render the BasicComponent, executing every lines of it.

But if you take a closer look at the test, you'll notice that we never assert for the text This is a basic component to be in the document.

It's not tested, however the code coverage will still be 100%.

I wanted to know if there was a way to find which lines are covered but not actually tested?

答案1

得分: 2

我想知道是否有办法找出哪些行被覆盖但实际上没有被测试?

我认为默认情况下是不行的。

Jest和大多数测试覆盖工具的工作方式是跟踪在测试运行期间执行的代码行,而不一定是哪些代码行在断言中被“测试”了。

你可以尝试遵循最佳实践,比如在“11个Jest最佳实践以充分利用您的测试”中提到的那些,来使用代码覆盖工具。

但你还可以将代码覆盖工具与变异测试相结合,使用像Stryker Mutator这样的框架:它会故意在你的代码中引入小错误(“变异”),然后运行你的测试来查看它们是否捕获到了这些错误。如果一行代码可以更改而不导致任何测试失败,那就意味着该行代码没有足够的测试。

这将有助于检测到您错过的关于文本“这是一个基本组件”的断言。

使用Stryker,一个“变异体”是应用程序中已经以一种小的方式更改的代码片段,如果你的测试仍然通过了这个更改,那么Stryker将将其标识为“存活的变异体”。

当应用于React组件时,Stryker将对组件的代码进行各种更改,并重新运行您的测试,以查看它们是否仍然通过。例如,它可以将字符串'That is a basic component'更改为'That is not a basic component'或删除渲染该文本的代码行。如果这些更改后仍然通过测试,Stryker将报告您的组件有存活的变异体,并指示存活的具体变异。

由于即使更改文本(“变异”)后您的测试仍然通过,因此您将知道测试是“不完整的”(缺少断言)。

然而,Limon Monte评论中指出stryker-mutator/stryker-js问题4375:“在JSX/TSX中变异字符串节点的可能性”。

Skyler有时会错误地将被测试覆盖的代码行标识为未覆盖,尽管它们实际上已被测试覆盖。这是因为它使用静态分析方法来识别未覆盖的代码行,而这种方法有时会不准确。

但是,Skyler的问题并不是不使用它的理由。它仍然是一个用于识别未覆盖的代码行的有价值工具,而错误识别的问题相对较少。

请注意,Skyler的问题:

  • 不限于Jest。它可以在使用Jest的任何项目中不正确地标识未覆盖的代码行。
  • 不是安全漏洞。该问题不影响代码的正确性,只影响Skyler报告的准确性。
  • 正在被Stryker团队跟踪。团队正在努力修复这个问题,但尚无修复的估计时间。
英文:

> I wanted to know if there was a way to find which lines are covered but not actually tested?

I do not believe so out of the box.

The way Jest and most test coverage tools work is by tracking which lines of code get executed during the test run, not necessarily which lines of code get "tested" in the sense of being included in an assertion.

All you can try (using a code coverage tool alone) is followed best practices, like the ones mentioned in "11 Jest Best Practices to Get the Most Out of Your Tests", from Ferenc Almasi.

But you can also combine a code coverage too with mutation testing, using a framework like Stryker Mutator: it will deliberately introduce small bugs ("mutations") into your code and running your tests to see if they catch the bugs. If a line of code can be changed without causing any test to fail, that is a sign that line is not adequately tested.

That would help detect your missed assertion for the text This is a basic component to be in the document.

With Stryker, a "mutant" is a piece of code in your application that has been changed in one small way, and if your tests still pass with that change, then Stryker would identify that as a 'surviving mutant'.

When applied to a React component, Stryker will make a variety of changes to the component's code and rerun your tests to see if they still pass.
For example, it could change the string &#39;That is a basic component&#39; to &#39;That is not a basic component&#39; or delete the line of code that renders that text.
If your tests still pass with these changes, Stryker will report that your component has surviving mutants and indicate the specific mutations that survived.

Since your test would still pass even when the text is changed ("mutated"), you will know that the test is "incomplete" (missing assertion).


However, Limon Monte points out in the comments to stryker-mutator/stryker-js issue 4375: "Possibility to mutate string nodes in JSX/TSX"

Skyler can sometimes incorrectly identify lines of code as uncovered when they are actually covered by tests.
This is because it uses a static analysis approach to identify uncovered lines of code, and this approach can sometimes be inaccurate.

However, the issue with Skyler is not a reason to not use it. It is still a valuable tool for identifying uncovered lines of code, and the issue with incorrect identification is relatively rare.

Note, The issue with Skyler:

  • is not specific to Jest. It can incorrectly identify uncovered lines of code in any project that uses Jest.
  • is not a security vulnerability. The issue does not affect the correctness of the code, it just affects the accuracy of the Skyler report.
  • is being tracked by the Stryker team. The team is working on a fix for the issue, but there is no ETA for the fix yet.

huangapple
  • 本文由 发表于 2023年7月12日 22:44:57
  • 转载请务必保留本文链接:https://go.coder-hub.com/76671819.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定