How to merge two lists of objects based on properties and merge duplicates into new object

Viewed 1730

I'm struggling to find a simple solution to my problem: I have two lists of objects and want to compare them based on one property (Serial) and create a new list that contains objects from both lists. If the object is only in list one i want to mark it as deleted (Status), if its only in list two it should be marked as new (Status). if its in both i want to mark it as changed (Status) and store the old and new value (Amount/ NewAmount).

So it looks something like this:

List One:

[
    { 
        serial: 63245-8,
        amount:  10
    },
    { 
        serial: 08657-5,
        amount:  100
    }
    ,
    { 
        serial: 29995-0,
        amount:  500
    }
]

List Two:

[
    { 
        serial: 63245-8,
        amount:  100
    },
    { 
        serial: 67455-1,
        amount:  100
    }
    ,
    { 
        serial: 44187-10,
        amount:  50
    }
]

Output:

[
    { 
        serial: 63245-8,
        amount:  10,
        newAmount:  100
        status: "changed"
    },
    { 
        serial: 08657-5,
        amount:  100
        status: "deleted"
    },
    { 
        serial: 29995-0,
        amount:  500,
        status: "deleted"
    }
    { 
        serial: 67455-1,
        amount:  100
        status: "new"
    }
    ,
    { 
        serial: 44187-10,
        amount:  50
        status: "new"
    }
]

I can't think of any good solution other than iterating over both lists and comparing with the other, building three different lists and merging them int the end and eventually even sorting them. I'm pretty sure there is a better solution to this, maybe even with AutoMapper ? Can anyone help me ?

Thanks!

Edit: Because the Question came up in the Comments. If Items are in both lists the Status can either be "changed" or "unchanged". It doesn't really matter for the implementation since i display the objects with their old and new amount and only want specially mark deleted and new objects. the Status "unchanged" would be a nice to have for future reference though.

4 Answers

This is a two-way comparison of lists, which you can achieve by using Linq's IEnumerable.Except() and IEnumerable.Intersect().

The first thing you should do is to write a class to hold the data items:

sealed class Data
{
    public string Serial { get; }
    public int    Amount { get; }

    public Data(string serial, int amount)
    {
        Serial = serial;
        Amount = amount;
    }
}

The next thing you need to do is to write an IEqualityComparer<T> that you can use to compare the items (you will need this to use Intersect() and Except():

sealed class DataComparer : IEqualityComparer<Data>
{
    public bool Equals(Data x, Data y)
    {
        return x.Serial.Equals(y.Serial);
    }

    public int GetHashCode(Data obj)
    {
        return obj.Serial.GetHashCode();
    }
}

Now write a class to receive the comparison data:

enum ComparisonState
{
    Unchanged,
    Changed,
    New,
    Deleted
}

sealed class ComparedData
{
    public Data            Data            { get; }
    public int             PreviousAmount  { get; }
    public ComparisonState ComparisonState { get; }

    public ComparedData(Data data, ComparisonState comparisonState, int previousAmount)
    {
        Data            = data;
        ComparisonState = comparisonState;
        PreviousAmount  = previousAmount;
    }

    public override string ToString()
    {
        if (ComparisonState == ComparisonState.Changed)
            return $"Serial: {Data.Serial}, Amount: {PreviousAmount}, New amount: {Data.Amount}, Status: Changed";
        else
            return $"Serial: {Data.Serial}, Amount: {Data.Amount}, Status: {ComparisonState}";
    }
}

(I've added a ToString() to that class for convenience.)

Now you can use Linq as follows. Read the comments to see how it works:

class Program
{
    public static void Main()
    {
        var list1 = new List<Data>
        {
            new Data("63245-8",  10),
            new Data("08657-5", 100),
            new Data("29995-0", 500),
            new Data("12345-0",  42)
        };

        var list2 = new List<Data>
        {
            new Data("63245-8", 100),
            new Data("12345-0",  42),
            new Data("67455-1", 100),
            new Data("44187-10", 50),
        };

        var comparer = new DataComparer();

        var newItems     = list2.Except(list1, comparer);    // The second list without items from the first list = new items.
        var deletedItems = list1.Except(list2, comparer);    // The first list without items from the second list = deleted items.
        var keptItems    = list2.Intersect(list1, comparer); // Items in both lists = kept items (but note: Amount may have changed).

        List<ComparedData> result = new List<ComparedData>();

        result.AddRange(newItems    .Select(item => new ComparedData(item, ComparisonState.New,     0)));
        result.AddRange(deletedItems.Select(item => new ComparedData(item, ComparisonState.Deleted, 0)));

        // For each item in the kept list, determine if it changed by comparing it to the first list.
        // Note that the "list1.Find()` is an O(N) operation making this quite slow.
        // You could speed it up for large collections by putting list1 into a dictionary and looking items up in it -
        // but this is unlikely to be needed for smaller collections.

        result.AddRange(keptItems.Select(item =>
        {
            var previous = list1.Find(other => other.Serial == item.Serial);
            return new ComparedData(item, item.Amount == previous.Amount ? ComparisonState.Unchanged : ComparisonState.Changed, previous.Amount);
        }));

        // Print the result, for illustration.

        foreach (var item in result)
            Console.WriteLine(item);
    }
}

The output of this is as follows:

Serial: 67455-1, Amount: 100, Status: New
Serial: 44187-10, Amount: 50, Status: New
Serial: 08657-5, Amount: 100, Status: Deleted
Serial: 29995-0, Amount: 500, Status: Deleted
Serial: 63245-8, Amount: 10, New amount: 100, Status: Changed
Serial: 12345-0, Amount: 42, Status: Unchanged

DotNet fiddle is here

I suggest you create a couple of custom classes to ease the code comprehension

public class Item
{
    public string serial;
    public int? amount;
    public int? newAmount;
    public string status;
}

public class L1Item : Item
{       
    public L1Item(string s, int a)
    {
        serial = s;
        amount = a;
        status = "deleted";
    }
}

public class L2Item : Item
{
    public L2Item(string s, int a)
    {
        serial = s;
        amount = a;
        status = "new";
    }
}

Then using the input you provided you can create two separate lists

List<Item> l1 = new List<Item>() { new L1Item("63245-8", 10), new L1Item("08657-5", 100), new L1Item("29995-0", 500) };
List<Item> l2 = new List<Item>() { new L2Item("63245-8", 100), new L2Item("67455-1", 100), new L2Item("44187-10", 50) };

Then you can concatenate them into a single list and group them by serial

var groupedList = l1.Concat(l2).GroupBy(x => x.serial);

Finally, having grouped all the items for each serial, make the corresponding changes and retrieve them.

var output = groupedList.Select(g => new Item()
{
    serial = g.Key,
    amount = g.First().amount,
    newAmount = g.Count() > 1 ? g.Last().amount : null,
    status = g.Count() > 1 ? "changed" : g.First().status
});

This is an example of a possible implementation

public class Obj
{
    public string serial { get; set; }
    public int amount { get; set; }
    public int? newAmount { get; set; }
    public Status status { get; set; }
}

public enum Status
{
    undefined,
    changed,
    deleted,
    @new
}
static void Main(string[] args)
    {
        string listOneJson = @"[
                                { 
                                    serial: '63245-8',
                                    amount:  10
                                },
                                { 
                                    serial: '08657-5',
                                    amount:  100
                                }
                                ,
                                { 
                                    serial: '29995-0',
                                    amount:  500
                                }
                            ]";
        string listTwoJson = @"[
                                {
                                    serial: '63245-8',
                                    amount: 100
                                },
                                {
                                    serial: '67455-1',
                                    amount: 100
                                }
                                ,
                                {
                                    serial: '44187-10',
                                    amount: 50
                                }
                               ]";
        IList<Obj> listOne = JsonConvert.DeserializeObject<IList<Obj>>(listOneJson);
        IList<Obj> listTwo = JsonConvert.DeserializeObject<IList<Obj>>(listTwoJson);

        var result = merge(listOne, listTwo);
    }

 public static IEnumerable<Obj> merge(IList<Obj> listOne, IList<Obj> listTwo)
 {

        List<Obj> allElements = new List<Obj>();
        allElements.AddRange(listOne);
        allElements.AddRange(listTwo);

        IDictionary<string, int> dict1 = listOne.ToDictionary(x => x.serial, x => x.amount);
        IDictionary<string, int> dict2 = listTwo.ToDictionary(x => x.serial, x => x.amount);
        IDictionary<string, Obj> dictResults = new Dictionary<string, Obj>();

        foreach (var obj in allElements)
        {
            string serial = obj.serial;

            if (!dictResults.ContainsKey(serial))
            {
                bool inListOne = dict1.ContainsKey(serial);
                bool inListTwo = dict2.ContainsKey(obj.serial);

                Obj result = new Obj { serial = serial };

                if (inListOne && inListTwo) {
                    result.status = Status.changed;
                    result.amount = dict1[serial];
                    result.newAmount = dict2[serial];
                }
                else if (!inListOne && inListTwo)
                {
                    result.status = Status.@new;
                    result.amount = dict2[serial];
                }
                else if (inListOne && !inListTwo)
                {
                    result.status = Status.deleted;
                    result.amount = dict1[serial];
                }

                dictResults.Add(serial, result);
            }
        }
        return dictResults.Values;
   }

There are a few good answers already. I just want to add a way to do this that doesn't increase the time complexity much when the data set increases. As a reminder, what Linq does behind the scene is just for loops.

Since you have to compare for 2 different conditions for each object in both lists, you have to iterate through each of them unfortunately. But there is a way to make the process faster.

Assuming you have n objects in listOne and m objects in listTwo.

You can first loop through all the objects in listOne and listTwo separately, and create a Dictionary for each of the list. i.e. dictOne and dictTwo. This will take a O(n) and O(m) time complexity respectively.

Then, loop through listOne and check if the items exist in dictTwo. Next, loop through listTwo and check if the items exist in dictOne.

That way the overall time complexity would be approximately O(n+m).

Data models:

public class InputData{
    public InputData(string serial, int amount){
        this.Serial = serial;
        this.Amount = amount;
    }
    public string Serial {get; set;}
    public int Amount{get;set;}
}

public class ResultData{
    public ResultData(string serial, int amount, int newAmount, string status){
        this.Serial = serial;
        this.Amount = amount;
        this.NewAmount = newAmount;
        this.Status = status;
    }

    public ResultData(string serial, int newAmount, string status){
        this.Serial = serial;
        this.NewAmount = newAmount;
        this.Status = status;
    }
    public string Serial {get; set;}
    public int Amount{get;set;}
    public int NewAmount{get;set;}
    public string Status {get;set;}
}

Main method:

public static void Main()
{
    List<InputData> listOne = new List<InputData>
    {
        new InputData("63245-8", 10),
        new InputData("08657-5", 100),
        new InputData("29995-0", 500)
    };

    List<InputData> listTwo = new List<InputData>
    {
        new InputData("63245-8", 100),
        new InputData("67455-1", 100),
        new InputData("44187-10", 50)
    };

    Dictionary<string, InputData> dictOne = CreateDictionary(listOne);      
    Dictionary<string, InputData> dictTwo = CreateDictionary(listTwo);

    List<ResultData> result = new List<ResultData>();

    result.AddRange(ProcessData(listOne, dictTwo, "deleted"));
    result.AddRange(ProcessData(listTwo, dictOne, "new"));

    foreach(var item in result){
        Console.WriteLine($"Serial: {item.Serial}, Amount: {item.Amount}, Status: {item.Status}");
    }
}

Result:

Serial: 63245-8, Amount: 100, Status: changed
Serial: 08657-5, Amount: 100, Status: deleted
Serial: 29995-0, Amount: 500, Status: deleted
Serial: 63245-8, Amount: 10, Status: changed
Serial: 67455-1, Amount: 100, Status: new
Serial: 44187-10, Amount: 50, Status: new

Here's my Fiddle if you want to take a look of the actual code.

Related