EF Core RemoveRange System.InvalidCastException : Unable to cast object of type 'System.Int32' to type 'System.Int64'

In a method to do database cleanup between integration tests I get exceptions for a couple tests when I remove either of the SaveChanges() calls in the code snippet below but if both are present it works fine.

System.InvalidCastException : Unable to cast object of type 'System.Int32' to type 'System.Int64'.
... A bunch of tables
           ctx.RemoveRange(ctx.RoleXPermission);
            ctx.SaveChanges();
            ctx.RemoveRange(ctx.UsersXAuthorizationGroup);
            ctx.SaveChanges();
            ctx.RemoveRange(ctx.AuthorizationGroupXRole);
... A couple more tables

The stack trace is

ValueComparer`1.Equals(Object left, Object right)
    CompositeCustomComparer.Equals(Object[] x, Object[] y)
    KeyValueIndex`1.Equals(KeyValueIndex`1 other)
    KeyValueIndex`1.Equals(Object obj)
    Dictionary`2.FindValue(TKey key)
    CommandBatchPreparer.AddUniqueValueEdges(Multigraph`2 commandGraph)
    CommandBatchPreparer.TopologicalSort(IEnumerable`1 commands)
    CommandBatchPreparer.BatchCommands(IList`1 entries, IUpdateAdapter updateAdapter)+MoveNext()
    BatchExecutor.Execute(IEnumerable`1 commandBatches, IRelationalConnection connection)
    RelationalDatabase.SaveChanges(IList`1 entries)
    StateManager.SaveChanges(IList`1 entriesToSave)
    StateManager.SaveChanges(DbContext _, Boolean acceptAllChangesOnSuccess)
    SqlServerExecutionStrategy.Execute[TState,TResult](TState state, Func`3 operation, Func`3 verifySucceeded)
    StateManager.SaveChanges(Boolean acceptAllChangesOnSuccess)
    DbContext.SaveChanges(Boolean acceptAllChangesOnSuccess)
    DbContext.SaveChanges()
    TestBase.GetCleanedTestContext() line 235
    TestBase.SetUp() line 48

I'm still trying to figure out exactly what this means and how I got here but when I debug this is what I see

            public bool Equals(object[] x, object[] y)
            {
                if (ReferenceEquals(x, y))
                {
                    return true;
                }

                if (x.Length != y.Length)
                {
                    return false;
                }

                for (var i = 0; i < x.Length; i++)
                {
                    if (!_equals[i](x[i], y[i]))
                    {
                        return false;
                    }
                }

                return true;
            }

Watch window while debugging

Digging a bit further at the stack line (after loading symbols) System.Private.CoreLib.dll!System.Collections.Generic.Dictionary<Microsoft.EntityFrameworkCore.Update.Internal.IKeyValueIndex, System.Collections.Generic.List<Microsoft.EntityFrameworkCore.Update.ModificationCommand>>.FindValue(Microsoft.EntityFrameworkCore.Update.Internal.IKeyValueIndex key) Line 402

I see

entry = ref entries[i];
if (entry.hashCode == hashCode && defaultComparer.Equals(entry.key, key))
{
    goto ReturnFound;
}

and for some reason entry.key._keyValues is object[] with one int and one long while key._keyValues is object[] with two longs.

I've review the data model and migrations and can't find anywhere that something went from long, long to int, long or vice-versa that could have made things inconsistent but I might have missed something so pointers on how to check for that are also welcome.