Why is my C# extension method used in a linq-to-entity query loading the database items into memory instead of running on the DB server?

Viewed 102

I created the following simple extension method:

public static IQueryable<TSource> DistinctBy<TSource, TKey>(this IQueryable<TSource> source, Expression<Func<TSource, TKey>> keySelector)
{
    return source.GroupBy(keySelector)
                 .Select(g => g.FirstOrDefault())
                 .AsQueryable();
}

My problem seems to be that the code actually loads the database rows into memory and tries to perform the operations in memory instead of doing what Linq-To-Entities is supposed to do and translate the query to SQL.

One can see above that my extension method does nothing that couldn't be translated to SQL, so why isn't it doing that?

The following two lines of code should be equivalent but the first one executes in <200ms while the other one runs for about 20 minutes before throwing an out of memory exception:

// Executes in less than 200ms.
var test_1 = dbContext.SomeTable.GroupBy(row => new { row.SomeColumn, row.SomeOtherColumn })
                                .Select(g => g.FirstOrDefault())
                                .Tolist();
// The line below takes some 20 minutes to eventually throw an out of memory exception.
// Notice I'm not even calling ToList().
var test_2 = dbContext.SomeTable.DistinctBy(row => new { row.SomeColumn, row.SomeOtherColumn });

Note that this extension method is just an example. It's relatively easy to just not use it and instead use the code defined inside the method directly. I'm saying this because I'm not asking how to make a "DistinctBy" method. Instead, what I'm asking is how to code an extension method that only uses other 'translatable' methods to avoid code repetition.

I understand that I can only use the extension methods provided by the EF provider but that is, in essence, what I'm doing. Can't I let it know that my method can be directly translated to SQL, how does the GroupBy method manage to do it?

0 Answers
Related