Executing in new AppDomain: FileLoadException for third party dll which loads "wrong version"?

UPDATE: I just was made aware that AppDomains are no longer supported in .NET core and .NET 5+. Maybe the solution we used in the passed simply no longer is possible.

We're doing something a bit awkward. We are trying to build an integration test framework for our GUI application using nunit. Since I am not allowed to run a new instance of the GUI in different tests in the same test suite (something to do with WPF), I am forced to run the GUI app in a different AppDomain.

This works, until the code start executing third party dlls which have to be loaded as well. For some weird reason, it no longer can find the correct version of the dll and throws a FileLoadException.

When I just remove all the AppDomain stuff, and run the test it does find the correct assemblies. But, as said, I am forced to use the AppDomain when I want to run all kinds of these GUI tests in the same test suite.

Any ideas why the usage of AppDomain results in looking for a different version of the DLL? Any clues how to get this resolved?

My test code:

using IAI.BMOne.UI;
using IAI.SystemControlling;
using NUnit.Framework;
using System;
using System.Threading;

namespace SomeNamespace
{
    [TestFixture, Apartment(ApartmentState.STA), Serializable]
    [RequiresThread]
    [TestFixture]
    public class ExperimentTest
    {
        private AppDomain systemControlAppDomain;

        [Test]
        public void Test()
        {
            var setup = new AppDomainSetup { ApplicationBase = AppDomain.CurrentDomain.BaseDirectory };
            systemControlAppDomain = AppDomain.CreateDomain("StartAndInitializeSystemControl domain", null, setup);

            void Action()
            {
                var application = new SystemControlApplication();
                var systemControl = new SystemControl(application, "279", false, PathsFactory.CreateDefaultConfig());
                // This method instantiates a new UnitContainer. Unity depends on CommonServiceLocator
                // I can't step into the method, the FileLoadException is thrown before actually jumping into the method
                systemControl.Initialize(null); 
            }
            
            StartCrossAppDomainDelegate(Action);
        }
        
        private void StartCrossAppDomainDelegate(CrossAppDomainDelegate action)
        {
            try
            {
                systemControlAppDomain.DoCallBack(action);
            }
            catch (AssertionException)
            {
                // Rethrow AssertionException
                throw;
            }
            catch (Exception ex)
            {
                var msg = $"Caught an exception that crossed the appdomain : {ex}";
                Assert.Fail(msg);
            }
            finally
            {
                AppDomain.Unload(systemControlAppDomain);
            }
        }        
    }
}

The full exception text

System.IO.FileLoadException: Could not load file or assembly 'CommonServiceLocator, Version=2.0.4.0, Culture=neutral, PublicKeyToken=489b6accfaf20ef0' or one of its dependencies. The located assembly's manifest definition does not match the assembly reference. (Exception from HRESULT: 0x80131040)
File name: 'CommonServiceLocator, Version=2.0.4.0, Culture=neutral, PublicKeyToken=489b6accfaf20ef0'
   at IAI.SystemControlling.SystemControl.Initialize(IAuditTrail auditTrail)
   at IAI.UnitTests.SystemSoftware.Application.Host.Tests.HostControllerTests.<Test>g__Action|1_0() in C:\dev\bookmaster\Source\UnitTests\Application\Host\HostControllerTests.cs:line 37
   at System.AppDomain.DoCallBack(CrossAppDomainDelegate callBackDelegate)

=== Pre-bind state information ===
LOG: DisplayName = CommonServiceLocator, Version=2.0.4.0, Culture=neutral, PublicKeyToken=489b6accfaf20ef0
 (Fully-specified)
LOG: Appbase = file:///C:/dev/bookmaster/Source/UnitTests/bin/x86/Debug/
LOG: Initial PrivatePath = NULL
Calling assembly : Unity.ServiceLocation, Version=5.11.1.0, Culture=neutral, PublicKeyToken=489b6accfaf20ef0.
===
LOG: This bind starts in default load context.
LOG: Using application configuration file: C:\Users\bpr\AppData\Local\JetBrains\Rider\r2r\2022.2.2.0.0R\E0B6E1074A66EAD81756C41426485E2\TestRunner\net461\ReSharperTestRunner32.exe.Config
LOG: Using host configuration file: 
LOG: Using machine configuration file from C:\Windows\Microsoft.NET\Framework\v4.0.30319\config\machine.config.
LOG: Post-policy reference: CommonServiceLocator, Version=2.0.4.0, Culture=neutral, PublicKeyToken=489b6accfaf20ef0
LOG: Attempting download of new URL file:///C:/dev/bookmaster/Source/UnitTests/bin/x86/Debug/CommonServiceLocator.DLL.
WRN: Comparing the assembly name resulted in the mismatch: Build Number
ERR: Failed to complete setup of assembly (hr = 0x80131040). Probing terminated.