LINQ's Distinct() on a particular property

Viewed 1181642

I am playing with LINQ to learn about it, but I can't figure out how to use Distinct when I do not have a simple list (a simple list of integers is pretty easy to do, this is not the question). What I if want to use Distinct on a list of an Object on one or more properties of the object?

Example: If an object is Person, with Property Id. How can I get all Person and use Distinct on them with the property Id of the object?

Person1: Id=1, Name="Test1"
Person2: Id=1, Name="Test1"
Person3: Id=2, Name="Test2"

How can I get just Person1 and Person3? Is that possible?

If it's not possible with LINQ, what would be the best way to have a list of Person depending on some of its properties in .NET 3.5?

23 Answers

What if I want to obtain a distinct list based on one or more properties?

Simple! You want to group them and pick a winner out of the group.

List<Person> distinctPeople = allPeople
  .GroupBy(p => p.PersonId)
  .Select(g => g.First())
  .ToList();

If you want to define groups on multiple properties, here's how:

List<Person> distinctPeople = allPeople
  .GroupBy(p => new {p.PersonId, p.FavoriteColor} )
  .Select(g => g.First())
  .ToList();

Note: Certain query providers are unable to resolve that each group must have at least one element, and that First is the appropriate method to call in that situation. If you find yourself working with such a query provider, FirstOrDefault may help get your query through the query provider.

Note2: Consider this answer for an EF Core (prior to EF Core 6) compatible approach. https://stackoverflow.com/a/66529949/8155

EDIT: This is now part of MoreLINQ.

What you need is a "distinct-by" effectively. I don't believe it's part of LINQ as it stands, although it's fairly easy to write:

public static IEnumerable<TSource> DistinctBy<TSource, TKey>
    (this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
    HashSet<TKey> seenKeys = new HashSet<TKey>();
    foreach (TSource element in source)
    {
        if (seenKeys.Add(keySelector(element)))
        {
            yield return element;
        }
    }
}

So to find the distinct values using just the Id property, you could use:

var query = people.DistinctBy(p => p.Id);

And to use multiple properties, you can use anonymous types, which implement equality appropriately:

var query = people.DistinctBy(p => new { p.Id, p.Name });

Untested, but it should work (and it now at least compiles).

It assumes the default comparer for the keys though - if you want to pass in an equality comparer, just pass it on to the HashSet constructor.

You could also use query syntax if you want it to look all LINQ-like:

var uniquePeople = from p in people
                   group p by new {p.ID} //or group by new {p.ID, p.Name, p.Whatever}
                   into mygroup
                   select mygroup.FirstOrDefault();

Starting with .NET 6, there is new solution using the new DistinctBy() extension in Linq, so we can do:

var distinctPersonsById = personList.DistinctBy(x => x.Id);

The signature of the DistinctBy method:

// Returns distinct elements from a sequence according to a specified
// key selector function.
public static IEnumerable<TSource> DistinctBy<TSource, TKey> (
    this IEnumerable<TSource> source,
    Func<TSource, TKey> keySelector);

I've written an article that explains how to extend the Distinct function so that you can do as follows:

var people = new List<Person>();

people.Add(new Person(1, "a", "b"));
people.Add(new Person(2, "c", "d"));
people.Add(new Person(1, "a", "b"));

foreach (var person in people.Distinct(p => p.ID))
    // Do stuff with unique list here.

Here's the article (now in the Web Archive): Extending LINQ - Specifying a Property in the Distinct Function

You can use DistinctBy() for getting Distinct records by an object property. Just add the following statement before using it:

using Microsoft.Ajax.Utilities;

and then use it like following:

var listToReturn = responseList.DistinctBy(x => x.Index).ToList();

where 'Index' is the property on which i want the data to be distinct.

You can do it (albeit not lightning-quickly) like so:

people.Where(p => !people.Any(q => (p != q && p.Id == q.Id)));

That is, "select all people where there isn't another different person in the list with the same ID."

Mind you, in your example, that would just select person 3. I'm not sure how to tell which you want, out of the previous two.

Override Equals(object obj) and GetHashCode() methods:

class Person
{
    public int Id { get; set; }
    public int Name { get; set; }

    public override bool Equals(object obj)
    {
        return ((Person)obj).Id == Id;
        // or: 
        // var o = (Person)obj;
        // return o.Id == Id && o.Name == Name;
    }
    public override int GetHashCode()
    {
        return Id.GetHashCode();
    }
}

and then just call:

List<Person> distinctList = new[] { person1, person2, person3 }.Distinct().ToList();

You should be able to override Equals on person to actually do Equals on Person.id. This ought to result in the behavior you're after.

If you use old .NET version, where the extension method is not built-in, then you may define your own extension method:

public static class EnumerableExtensions
{
    public static IEnumerable<T> DistinctBy<T, TKey>(this IEnumerable<T> enumerable, Func<T, TKey> keySelector)
    {
        return enumerable.GroupBy(keySelector).Select(grp => grp.First());
    }
}

Example of usage:

var personsDist = persons.DistinctBy(item => item.Name);

May be this could help, Try this. Using HashSet is more code performant.

public static IEnumerable<TSource> DistinctBy<TSource, TKey>(this IEnumerable<TSource> source, Func<TSource, TKey> keySelector)
{
    var known = new HashSet<TKey>();
    return source.Where(element => known.Add(keySelector(element)));
}

Please give a try with below code.

var Item = GetAll().GroupBy(x => x .Id).ToList();
Related