英文:
Running specific test case not triggering in nunit
问题
以下是您提供的翻译:
"The project I'm working on has a test project that contains a large set of system tests that run for about 1 hour each. The project has a single modular test function with about 40 TestCase
annotations. These tests are too heavy to do in parallel on a single computer so I'm trying to spread them out over multiple buildnodes (1 test per node).
A trimmed version of that code would be:
[TestCase("4.6", "20180001 V320")]
//more test cases
public void TestAutomaticLists(string version, string customer, string machine = "MachineConfig")
{
//test
}
To do this I compile the code and run the NunitConsoleRunner
to explore for tests:
nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --explore="tests.txt;format=cases" --noheader --nocolor --where "cat == AST"
This produces a file with all test cases. For example:
SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
Within github actions I then parse this output and use a matrix build to spread the tests over the nodes.
I know for sure that the nodes get a valid test case, Nunit also gets all the binaries it needs to run. But 0 tests get executed. The output of the following code
write-output "test to run: $test"
nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --test="$test"
is
test to run: SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
NUnit Console 3.16.3 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Thursday, July 6, 2023 2:14:03 PM
Runtime Environment
OS Version: Microsoft Windows NT 6.2.9200.0
Runtime: .NET Framework CLR v4.0.30319.42000
Test Files
Tests\SystemTests\bin\x64\Debug\SystemTests.dll
Test Filters
Test: SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)
Run Settings
DisposeRunners: True
WorkDirectory: D:\workspaces\vacam\vacam
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.7.2
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
TargetRuntimeFramework: net-4.7.2
NumberOfTestWorkers: 4
Test Run Summary
Overall result: Passed
Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2023-07-06 12:14:03Z
End time: 2023-07-06 12:14:05Z
Duration: 1.477 seconds
Results (nunit3) saved as TestResult.xml
I have ensured that the right binaries are in there by calling the --explore option again on the node executing the tests. The test I want to run does show up in the list. I'm guessing I'm calling the --test option incorrectly or something, but I'm not seeing it right now. Any help is appreciated!"
英文:
The project I'm working on has a test project that contains a large set of system tests that run for about 1 hour each. The project has a single modular test function with about 40 TestCase
annotations. These tests are to heavy to do in parallel on a single computer so I'm trying to spread them out over multiple buildnodes (1 test per node).
A trimmed version of that code would be:
[TestCase("4.6", "20180001 V320")]
//more test cases
public void TestAutomaticLists(string version, string customer, string machine = "MachineConfig")
{
//test
}
To do this I compile the code and run the NunitConsoleRunner
to explore for tests:
nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --explore="tests.txt;format=cases" --noheader --nocolor --where "cat == AST"
This produces a file with all test cases. For example:
SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
Within github actions I then parse this output and use a matrix build to spread the tests over the nodes.
I know for sure that the nodes get a valid test case, Nunit also gets all the binaries it needs to run. But 0 tests get executed. The output of the following code
write-output "test to run: $test"
nunit3-console.exe "Tests\SystemTests\bin\x64\Debug\SystemTests.dll" --test="$test"
is
test to run: SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
NUnit Console 3.16.3 (Release)
Copyright (c) 2022 Charlie Poole, Rob Prouse
Thursday, July 6, 2023 2:14:03 PM
Runtime Environment
OS Version: Microsoft Windows NT 6.2.9200.0
Runtime: .NET Framework CLR v4.0.30319.42000
Test Files
Tests\SystemTests\bin\x64\Debug\SystemTests.dll
Test Filters
Test: SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)
Run Settings
DisposeRunners: True
WorkDirectory: D:\workspaces\vacam\vacam
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.7.2
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
TargetRuntimeFramework: net-4.7.2
NumberOfTestWorkers: 4
Test Run Summary
Overall result: Passed
Test Count: 0, Passed: 0, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2023-07-06 12:14:03Z
End time: 2023-07-06 12:14:05Z
Duration: 1.477 seconds
Results (nunit3) saved as TestResult.xml
I have ensured that the right binaries are in there by calling the --explore option again on the node executing the tests. The test I want to run does show up in the list. I'm guessing I'm calling the --test option incorrectly or something, but I'm not seeing it right now. Any help is appreciated!
答案1
得分: 0
NUnit 被告知运行名为
SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)
而不是
SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
由于未找到该测试,它不会运行任何内容。
参数 $test 必须设置为包含引号,并且不能在传递给 NUnit 时附加额外的引号。这在命令行中很难实现,这就是为什么存在 --testlist 选项的原因。
我建议创建一个文件列表,就像你现在正在做的那样,然后将该文件拆分成你需要的数量。这样可以避免在命令行中出现额外的双引号问题。
英文:
NUnit is being told to run the test named
SystemTests.Tests.AutomaticListTests.TestAutomaticLists(4.6,20180001 V320)
Rather than
SystemTests.Tests.AutomaticListTests.TestAutomaticLists("4.6","20180001 V320")
Since that test is not found, it doesn't run anything.
The argument $test has to be set up with the quotes included and must not be passed to NUnit with any added quotes around it. This is very hard to accomplish from the command-line, which is why the --testlist option exists.
I suggest creating a list of files, as you are now doing, and then splitting that file into as many files as you need. That will avoid any problem with extra double quotes on the command-line.
通过集体智慧和协作来改善编程学习和解决问题的方式。致力于成为全球开发者共同参与的知识库,让每个人都能够通过互相帮助和分享经验来进步。
评论