Why Visual Studio doesn't run tests that have a method call?

I've built a test case for a NUnit test and every time i ran the tests, some were skipped. No error, no message... just didn't run, even though the test appear as complete. I have 625 cases but it only runs 81.

Test Explorer

My test case consistis in 5 string and 1 boolean, and the strings are generated by a method.

Test Case Before

Random String Method

After some investigations, i tried inserting the string previously generated instead of calling the methods(below) and it worked.

Test Case Aafter

So i was wandering, why i can't run the tests calling the method to create the string?

I tried updating Visual Studio, NUnit... Neither worked while the method to create the string was on the test case.

There is a mismatch between how VS runs tests and how NUnit expects to run them. Mostly this is taken care of in the NUnit3 VS adapter... 'cause that's what adapters do.

Part of the "adapting" that happens is that NUnit tests are discovered twice... once when VS discovery runs and once when the test is executed. Like this...

VS Discovery => NUnit Discovery

VS Execution => NUnit Discovery => NUnit Execution

Mostly, this works fine. But when test case discovery involves random generation of data, it happens twice, causing the test cases to change.

NUnit is designed for repeatability of tests and the adapter makes use of it. So if you use something like the RandomAttribute to generate values, the same values will be generated for both discovery and execution. That's because the adapter saves the random seed in the VS discovery phase and re-uses it in the execution phase.

But if the user generates random data in their own code, that's another matter. That code doesn't know about NUnit's requirements for repeatability. The way around this is to avoid use of the Random class and to use NUnit's internal random facilities.

The class Randomizer is used by NUnit internally but is also available for your own use. You could make use of it in your case in one of two ways:

1. Just use it instead of Random, from which it inherits. Everything will work the same but you'll get repeatability across the two discovery passes.

2. Experiment with Randomizer.GetString(), which will generate strings for you. That's what I would use.

In either case, you should NOT create your own instance of Randomizer. Instead, use the instance created by NUnit, which will be re-created in a repeatable way during the execution phase. It's available as TestContext.CurrentContext.Randomizer.