Debug visualizer won't work in tests when the object that is visualized is in another assembly

I've been trying to write a custom debugger visualizer where the visualizer (inheriting from DialogDebuggerVisualizer) is in one assembly and the object that I want to visualize in another. I want to use the visualizer from unit tests.

As long as both types are in the same assembly it works as expected and I get the magnifying glass next to my variable in the watch window. If I leave everything else the same, but move the type I want to visualize to its own assembly it stops working (i.e. no magnifying glass shows up in the watch window) from unit tests (tried from Visual Studio's runner, ReSharper and NCrunch).

The visualizer is implemented like this:

namespace Visualizer
{
    public class DebuggerSide : DialogDebuggerVisualizer
    {
        protected override void Show(IDialogVisualizerService windowService, IVisualizerObjectProvider objectProvider)
        {
            string message = objectProvider?.GetObject()?.ToString() ?? "Failed to get object data";
            MessageBox.Show(message);
        }
    }
}

and the object I want to visualize like this:

namespace ProjectWithObjectToVisualize
{
    [DebuggerVisualizer("Visualizer.DebuggerSide, Visualizer, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null")]
    [Serializable]
    public class ObjectToVisualize
    {
        public override string ToString()
        {
            return "Testing";
        }
    }
}

and the xUnit test as follow:

[Fact]
public void Test()
{
    var c = new ObjectToVisualize();
}

The console program works independently of which assembly I put the ObjectToVisualize in.

What should I change to make it work?