Entity Framework Core GroupBy - aggregate functions on related entities

In EF Core, when using GroupBy I cannot get aggregate functions to work on fields from related entities. Here is an example to illustrate what I mean:

And I am trying to run the following query:

var list = db.Loans
   .GroupBy(x => x.Book.Isbn)
   .Select(
      x => new LoanQueryResult
      {
         Isbn = x.Key,
         AverageAge = x.Average(y => y.Member.Age)   // note here that I am navigating to a related entity
      }
   )
   .ToList();

So the objective in the above query is, for each Book Isbn, I want the average member age of the Members who have borrowed it.

The error that Entity Framework Core returns is as follows:

The LINQ expression '(EntityShaperExpression: 
    EntityType: Loan
    ValueBufferExpression: 
        (ProjectionBindingExpression: EmptyProjectionMember)
    IsNullable: False
).Member.Age' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.

How can I get this working, on the assumption that the query needs to start out from the Loans table?

Here is my model (in case it helps):

public class Book
{
   public int Id { get; set; }
   public string Isbn { get; set; }
   public string Title { get; set; }

   public IList<Loan> Loans { get; set; }
}


public class Member
{
   public int Id { get; set; }
   public string FirstName { get; set; }
   public string Surname { get; set; }
   public int Age { get; set; }

   public IList<Loan> Loans { get; set; }
}


public class Loan
{
   public int Id { get; set; }
   public int BookId { get; set; }
   public int MemberId { get; set; }
   public DateTime StartDate { get; set; }
   public DateTime EndDate { get; set; }

   public Book Book { get; set; }
   public Member Member { get; set; }
}