How to get duplicate items from a list using LINQ?

user_336940 0
Viewed 194204

I'm having a List<string> like:

List<String> list = new List<String>{"6","1","2","4","6","5","1"};

I need to get the duplicate items in the list into a new list. Now I'm using a nested for loop to do this.

The resulting list will contain {"6","1"}.

Is there any idea to do this using LINQ or lambda expressions?

duplicates linq c#
9 Answers

I know it's not the answer to the original question, but you may find yourself here with this problem.

If you want all of the duplicate items in your results, the following works.

var duplicates = list
    .GroupBy( x => x )               // group matching items
    .Where( g => g.Skip(1).Any() )   // where the group contains more than one item
    .SelectMany( g => g );           // re-expand the groups with more than one item

In my situation I need all duplicates so that I can mark them in the UI as being errors.

Scott_Langham_118980

All mentioned solutions until now perform a GroupBy. Even if I only need the first Duplicate all elements of the collections are enumerated at least once.

The following extension function stops enumerating as soon as a duplicate has been found. It continues if a next duplicate is requested.

As always in LINQ there are two versions, one with IEqualityComparer and one without it.

public static IEnumerable<TSource> ExtractDuplicates(this IEnumerable<TSource> source)
{
    return source.ExtractDuplicates(null);
}
public static IEnumerable<TSource> ExtractDuplicates(this IEnumerable<TSource source,
    IEqualityComparer<TSource> comparer);
{
    if (source == null) throw new ArgumentNullException(nameof(source));
    if (comparer == null)
        comparer = EqualityCompare<TSource>.Default;

    HashSet<TSource> foundElements = new HashSet<TSource>(comparer);
    foreach (TSource sourceItem in source)
    {
        if (!foundElements.Contains(sourceItem))
        {   // we've not seen this sourceItem before. Add to the foundElements
            foundElements.Add(sourceItem);
        }
        else
        {   // we've seen this item before. It is a duplicate!
            yield return sourceItem;
        }
    }
}

Usage:

IEnumerable<MyClass> myObjects = ...

// check if has duplicates:
bool hasDuplicates = myObjects.ExtractDuplicates().Any();

// or find the first three duplicates:
IEnumerable<MyClass> first3Duplicates = myObjects.ExtractDuplicates().Take(3)

// or find the first 5 duplicates that have a Name = "MyName"
IEnumerable<MyClass> myNameDuplicates = myObjects.ExtractDuplicates()
    .Where(duplicate => duplicate.Name == "MyName")
    .Take(5);

For all these linq statements the collection is only parsed until the requested items are found. The rest of the sequence is not interpreted.

IMHO that is an efficiency boost to consider.

user_22817900
Related