Change behaviour of AutoFixture with AutoMoq to return false for methods

First of all, you may want to use AutoMoqDataAttribute to create a mock of the ITeam interface:

public class AutoMoqDataAttribute : AutoDataAttribute
{
    public AutoMoqDataAttribute()
        : base(new Fixture().Customize(new AutoMoqCustomization()))
    {
    }
}

There is no need in cusomizing fixture to configure your mocks. You'd better do that in the tests itself (the arrange section):

[Theory, AutoMoqData]
public void ShouldRosterToTeamWhenPlayerIsNotRostered(Player player, Mock<ITeam> mockedTeam)
{
    mockedTeam.Setup(t => t.HasPlayer(player)).Returns(false);
    player.RosterToTeam(mockedTeam.Object);
    mockedTeam.Verify(team => team.AddPlayer(player), Times.Once);
}

[Theory, AutoMoqData]
public void ShouldNotRosterToTeamWhenPlayerIsRostered(Player player, Mock<ITeam> mockedTeam)
{
    mockedTeam.Setup(t => t.HasPlayer(player)).Returns(true);
    player.RosterToTeam(mockedTeam.Object);
    mockedTeam.Verify(team => team.AddPlayer(player), Times.Never);
}

and, finaly, the simplified RoastToTeam implementation:

public class Player
{
    public void RosterToTeam(ITeam team)
    {
        if (team.HasPlayer(this))
        {
            return;
        }
        team.AddPlayer(this);
    }
}

Considering everything that was proposed so far, I believe the only other option is to implement your own custom AutoMoqCustomizaiton that will treat the methods returning bool as a special case and will omit setting them.

You could create a behavior that would reset the members to return false, but you would not be able to then customize your mocks to return any value other than false, since behaviors work at the moment AutoFixture has already created the value.

Creating or altering the setup for all members returning bool in a mock object by just using reflection is not an easy task. You'll have to copy a good chunk of the AutoMoq into your code base. Luckily for you the license under which we publish AutoFixture is the most permissive of all OSS licenses, so you can copy and distribute your custom version without any restrictions.

The part that you'll likely want to change is this line here, in MockVirtualMethodsCommand. Your version should look something like this.

foreach (var method in methods.Where(x => x.ReturnType != typeof(bool)))

If this was my code-base, in cases like this I would not use mocks at all, and would probably opt for fakes. Consider the same code you proposed but instead of using Moq, the test project would implement the ITeam interface in a local class.

public class FakeTeam : ITeam
{
    public List<IPlayer> Players { get; set; } = new();

    public bool HasPlayer(IPlayer player)
    {
       return this.Players.Contains(player);
    }

    public void AddPlayer(IPlayer player)
    {
       this.Players.Add(player);
    }
}

Now the same test that you wrote would look something like this.

[Theory, AutoData]
public void ShouldRosterToTeamWhenPlayerIsNotRostered(
    Player player, FakeTeam team)
{
    player.RosterToTeam(team);

    Assert.Contains(player, team.Players); 
}

Depending on your business you might even want to use the actual implementation of ITeam, and that would be fine.

As far I can see, AutoMoq resolves the returntype directly. (Source here)

Which means you can't make all method that returns bool to return false unless you inject the value (fixture.Inject(false)) - which then of course means that you'll get false each time you ask AF for a bool.

What I've done in your scenario, is to create an empty interface that inherits from the interface I want to modify the behavior for, and register it.

Like this:

...
public interface ITeamFake : ITeam {} // empty interface for modifying the mock
...

// Modify ITeam behavior
fixture.Register<Mock<ITeamFake>, Mock<ITeam>>(t => 
    {
        t.Setup(m => m.HasPlayer(It.IsAny<IPlayer>())).Returns(true);       
        return t.As<ITeam>();       
    });
       

This works for me anyways.