Static Data from xunit MemberData function is computed twice

Question

I'm having some trouble with computed data from a static class in a C# Xunit test being computed twice.

The actual production code this would be used for is much more complicated, but the code that follows is enough to exhibit the issue I am seeing.

In the code below I have a randomly generated, lazily loaded int seeded off of the current time.

All I am testing here is that this property is equal to itself. I insert the property's value into the test via a MemberData function.

Since the property ought to only be initialized once, I'd expect that this test should always pass. I would expect that the static field would be initialized when the RandomIntMemberData function is run and never again.

However, this test consistently fails. The value inserted into the test, and the value tested against are always different.

Further if I debug, I only see the initialization code being hit once. That is, the value being tested. I never see the initialization of the value being tested against.

Am I misunderstanding something, or is Xunit doing some weird behind the scenes magic to setup it's input data, then initializing the value again when the test is actually run?

Minimal Code to Reproduce Bug

public static class TestRandoIntStaticClass
{
    private static readonly Lazy LazyRandomInt = new Lazy(() =>
    {
        // lazily initialize a random interger seeded off of the current time
        // according to readings, this should happen only once
        return new Random((int) DateTime.Now.Ticks).Next();
    });

    // according to readings, this should be a thread safe operation
    public static int RandomInt => LazyRandomInt.Value; 
}

The Test

public class TestClass
{
    public static IEnumerable RandomIntMemberData()
    {
        var randomInt = new List
        {
            new object[] {TestRandoIntStaticClass.RandomInt},
        };

        return randomInt as IEnumerable;
    }

    [Theory]
    [MemberData(nameof(RandomIntMemberData))]
    public void RandoTest(int rando)
    {
        // these two ought to be equal if TestRandoIntStaticClass.RandomInt is only initialized once 
        Assert.True(rando == TestRandoIntStaticClass.RandomInt,
                    $"{nameof(rando)} = {rando} but {nameof(TestRandoIntStaticClass.RandomInt)} = {TestRandoIntStaticClass.RandomInt}");
    }
}

user_4147046 · Answer

At the time of tests discovery, Visual Studio Xunit console runner creates AppDomain with test data for all attributes like MemberData, ClassData, DataAttribute so all data are just saved in memory after build (that is also why XUnit require classes to be serializable).

We can verify this by adding a simple logger to your methods:

namespace XUnitTestProject1
{
    public class TestClass
    {
        public static IEnumerable RandomIntMemberData()
        {
            var randomInt = new List
            {
                new object[]
                    {TestRandoIntStaticClass.RandomInt},
            };
            return randomInt;
        }

        [Theory]
        [MemberData(nameof(RandomIntMemberData))]
        public void RandoTest(int rando)
        {
            // these two ought to be equal if TestRandoIntStaticClass.RandomInt is only initialized once 
            Assert.True(rando == TestRandoIntStaticClass.RandomInt, $"{nameof(rando)} = {rando} but {nameof(TestRandoIntStaticClass.RandomInt)} = {TestRandoIntStaticClass.RandomInt}");
        }

    }

    public static class TestRandoIntStaticClass
    {
        private static readonly Lazy LazyRandomInt = new Lazy(() =>
        {   // lazily initialize a random interger seeded off of the current time
            // according to readings, this should happen only once
            var randomValue = new Random((int) DateTime.Now.Ticks).Next();

            File.AppendAllText(@"D:\var\log.txt", $"Call TestRandoIntStaticClass {randomValue}; ThreadId {Thread.CurrentThread.ManagedThreadId} " + Environment.NewLine);
            return randomValue;
        });
       
        public static int RandomInt => LazyRandomInt.Value; // according to readings, this should be a thread safe operation
    }
}

As a result we see in logs:

> Call TestRandoIntStaticClass 1846311153; ThreadId 11  
> Call TestRandoIntStaticClass 1007825738; ThreadId 14

And in test execution result

rando = 1846311153 but RandomInt = 1007825738
Expected: True
Actual:   False
   at

However, if you use dotnet test will be successful because 'data generation' and test run will be launched on one process

Static Data from xunit MemberData function is computed twice

1 Answers