在NUnit中运行特定测试用例未触发。

huangapple go评论109阅读模式
英文:

Running specific test case not triggering in nunit

问题

以下是您提供的翻译:

"The project I'm working on has a test project that contains a large set of system tests that run for about 1 hour each. The project has a single modular test function with about 40 TestCase annotations. These tests are too heavy to do in parallel on a single computer so I'm trying to spread them out over multiple buildnodes (1 test per node).

A trimmed version of that code would be:

[TestCase("4.6", "20180001 V320")]
//more test cases
public void TestAutomaticLists(string version, string customer, string machine = "MachineConfig")
{
  //test
}

To do this I compile the code and run the NunitConsoleRunner to explore for tests:

nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --explore="tests.txt;format=cases" --noheader --nocolor --where "cat == AST"

This produces a file with all test cases. For example:

SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")

Within github actions I then parse this output and use a matrix build to spread the tests over the nodes.

I know for sure that the nodes get a valid test case, Nunit also gets all the binaries it needs to run. But 0 tests get executed. The output of the following code

write-output "test to run: $test"
nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --test="$test"

is

test to run: SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
NUnit Console 3.16.3 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Thursday, July 6, 2023 2:14:03 PM

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    Tests\SystemTests\bin\x64\Debug\SystemTests.dll

Test Filters
    Test: SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)


Run Settings
    DisposeRunners: True
    WorkDirectory: D:\workspaces\vacam\vacam
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.7.2
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    TargetRuntimeFramework: net-4.7.2
    NumberOfTestWorkers: 4

Test Run Summary
  Overall result: Passed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2023-07-06 12:14:03Z
    End time: 2023-07-06 12:14:05Z
    Duration: 1.477 seconds

Results (nunit3) saved as TestResult.xml

I have ensured that the right binaries are in there by calling the --explore option again on the node executing the tests. The test I want to run does show up in the list. I'm guessing I'm calling the --test option incorrectly or something, but I'm not seeing it right now. Any help is appreciated!"

英文:

The project I'm working on has a test project that contains a large set of system tests that run for about 1 hour each. The project has a single modular test function with about 40 TestCase annotations. These tests are to heavy to do in parallel on a single computer so I'm trying to spread them out over multiple buildnodes (1 test per node).

A trimmed version of that code would be:

[TestCase("4.6", "20180001 V320")]
//more test cases
public void TestAutomaticLists(string version, string customer, string machine = "MachineConfig")
{
  //test
}

To do this I compile the code and run the NunitConsoleRunner to explore for tests:

nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --explore="tests.txt;format=cases" --noheader --nocolor --where "cat == AST"

This produces a file with all test cases. For example:

SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")

Within github actions I then parse this output and use a matrix build to spread the tests over the nodes.

I know for sure that the nodes get a valid test case, Nunit also gets all the binaries it needs to run. But 0 tests get executed. The output of the following code

write-output "test to run: $test"
nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --test="$test"

is

test to run: SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
NUnit Console 3.16.3 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Thursday, July 6, 2023 2:14:03 PM

Runtime Environment
   OS Version: Microsoft Windows NT 6.2.9200.0
   Runtime: .NET Framework CLR v4.0.30319.42000

Test Files
    Tests\SystemTests\bin\x64\Debug\SystemTests.dll

Test Filters
    Test: SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)


Run Settings
    DisposeRunners: True
    WorkDirectory: D:\workspaces\vacam\vacam
    ImageRuntimeVersion: 4.0.30319
    ImageTargetFrameworkName: .NETFramework,Version=v4.7.2
    ImageRequiresX86: False
    ImageRequiresDefaultAppDomainAssemblyResolver: False
    TargetRuntimeFramework: net-4.7.2
    NumberOfTestWorkers: 4

Test Run Summary
  Overall result: Passed
  Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
  Start time: 2023-07-06 12:14:03Z
    End time: 2023-07-06 12:14:05Z
    Duration: 1.477 seconds

Results (nunit3) saved as TestResult.xml

I have ensured that the right binaries are in there by calling the --explore option again on the node executing the tests. The test I want to run does show up in the list. I'm guessing I'm calling the --test option incorrectly or something, but I'm not seeing it right now. Any help is appreciated!

答案1

得分: 0

NUnit 被告知运行名为

SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)

而不是

SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")

由于未找到该测试,它不会运行任何内容。

参数 $test 必须设置为包含引号,并且不能在传递给 NUnit 时附加额外的引号。这在命令行中很难实现,这就是为什么存在 --testlist 选项的原因。

我建议创建一个文件列表,就像你现在正在做的那样,然后将该文件拆分成你需要的数量。这样可以避免在命令行中出现额外的双引号问题。

英文:

NUnit is being told to run the test named

SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)

Rather than

SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")

Since that test is not found, it doesn't run anything.

The argument $test has to be set up with the quotes included and must not be passed to NUnit with any added quotes around it. This is very hard to accomplish from the command-line, which is why the --testlist option exists.

I suggest creating a list of files, as you are now doing, and then splitting that file into as many files as you need. That will avoid any problem with extra double quotes on the command-line.

huangapple
  • 本文由 发表于 2023年7月6日 21:09:25
  • 转载请务必保留本文链接:https://go.coder-hub.com/76629198.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定