ASP.NET Core 5 Blazor WASM, gRPC, Entity Framework Core 5: many-to-many results in stack overflow

Question

Info:

Using: Visual Studio v16.9.0 Preview 2.0 and .NET SDK 5.0.200
Kestrel hosted Blazor WASM / gRPC / EF Core project with SQLite DB.

Sample project: (full working source available at GitHub) it's a (simple) prototype of a blog cms with a many to many relationship between posts and tags.

When I try to return a list of posts with tags from BlogService.cs the application stops with an 'stack overflow' error. It looks to me like a reference loop, like what you get when you're using json. But maybe I'm wrong, I don't know. (With Json.Net you need to set ReferenceLoopHandling = ReferenceLoopHandling.Ignore to make many to many work)

Or can I make changes to the LINQ query so that .Include(tipd => tipd.TagsInPostData) in BlogService.cs only returns the tags in each post (one level deep) and doesn't try to resolve all of the posts in each tag and so on.

I'm still learning C#, EF Core, gRPC, LINQ and English is not my first language so I hope you understand my question. If not, please say so and I'll try to do better.

(part of) BlogService.cs, full source here

public override async Task GetPosts(Empty request, ServerCallContext context)
{
    var posts = new Posts();
    var allPosts = await dbContext.Posts
        .Where(ps => ps.PostStat == PostStatus.Published)
        .Include(pa => pa.PostAuthor)
        .Include(pe => pe.PostExt)
        //.Include(tipd => tipd.TagsInPostData) // TODO: [ERROR] / doesn't work: results in a stack overflow.
        .OrderByDescending(dc => dc.DateCreated)
        .ToListAsync();
    posts.PostsData.AddRange(allPosts);
    return posts;
}

The classes from which EF Core creates the tables (including the join table, EF Core creates that one 'automagically') are created from the protobuf file blog.proto. To make adding records to the join table work with gRPC I've asked a question at GitHub and I've modified my code accordingly in ApplicationDbContext.cs

(part of) ApplicationDbContext.cs, full source here

modelBuilder
    .Entity()
    .HasMany(e => e.TagsInPostData)
    .WithMany(e => e.PostsInTagData)
    .UsingEntity>(
        "PostsTags", // (Join) Table Name.(Renames EF Core autogenerated 'PostTag' table to 'PostsTags' table)
        b => b.HasOne().WithMany().HasForeignKey("TagId"), // Field Name.
        b => b.HasOne().WithMany().HasForeignKey("PostId") // Field Name.
    );

// Adding data to Many to Many Join Table 'PostsTags' now works because of just the 2 lines below.
// See: https://github.com/dotnet/efcore/issues/23703#issuecomment-758801618
modelBuilder.Entity().Navigation(e => e.TagsInPostData).HasField("tagsInPostData_");
modelBuilder.Entity().Navigation(e => e.PostsInTagData).HasField("postsInTagData_");

(part of) blog.proto, full source here

message Post { // For Public Access
    int32 post_id = 1;
    int32 author_id = 2;
    string title = 3;
    string date_created = 4; // DateTime (UTC) string because of SQLite
    PostStatus post_stat = 5; // enum
    PostExtended post_ext = 6; // one to one
    Author post_author = 7; // Post with one author, one to one
    repeated Tag tags_in_post_data = 8; // Post with many Tags
}
message Posts {
    repeated Post posts_data = 1;
}

/*
    Many to Many Tags in auto generated table "PostsTags"
    EF Core auto generates 'PostTag' table, Renaming it to 'PostsTags' is done in ApplicationDbContext
    Because of "message Post" with "repeated Tag tags_data"
    and "message Tag" with "repeated Post posts_data"
    EF Core creates "PostTag" table 'automagically'.
*/

message Tag {
    //string tag_id = 1; // Tag itself: string
    int32 tag_id = 1;
    string name = 2;
    repeated Post posts_in_tag_data = 3; // Tag with many Posts
}
message Tags {
    repeated Tag tags_data = 1;
}

Error:

info: 15-1-2021 14:08:44.437 CoreEventId.ContextInitialized[10403] (Microsoft.EntityFrameworkCore.Infrastructure)
      Entity Framework Core 5.0.1 initialized 'ApplicationDbContext' using provider 'Microsoft.EntityFrameworkCore.Sqlite' with options: SensitiveDataLoggingEnabled
info: 15-1-2021 14:08:45.908 RelationalEventId.CommandExecuted[20101] (Microsoft.EntityFrameworkCore.Database.Command)
      Executed DbCommand (4ms) [Parameters=[], CommandType='Text', CommandTimeout='30']
      SELECT "p"."PostId", "p"."AuthorId", "p"."DateCreated", "p"."PostStat", "p"."Title", "a"."AuthorId", "a"."DateCreated", "a"."Name", "p0"."PostId", "p0"."Content", "p0"."Ts", "t0"."PostId", "t0"."TagId", "t0"."TagId0", "t0"."Name"
      FROM "Posts" AS "p"
      INNER JOIN "Authors" AS "a" ON "p"."AuthorId" = "a"."AuthorId"
      LEFT JOIN "PostsExtented" AS "p0" ON "p"."PostId" = "p0"."PostId"
      LEFT JOIN (
          SELECT "p1"."PostId", "p1"."TagId", "t"."TagId" AS "TagId0", "t"."Name"
          FROM "PostsTags" AS "p1"
          INNER JOIN "Tags" AS "t" ON "p1"."PostId" = "t"."TagId"
      ) AS "t0" ON "p"."PostId" = "t0"."TagId"
      WHERE "p"."PostStat" = 1
      ORDER BY "p"."DateCreated" DESC, "p"."PostId", "a"."AuthorId", "p0"."PostId", "t0"."PostId", "t0"."TagId", "t0"."TagId0"
Stack overflow.
   at System.Text.UTF8Encoding.GetByteCount(System.String)
   at Google.Protobuf.CodedOutputStream.ComputeStringSize(System.String)
   at BlazorWasmGrpcBlog.Shared.Protos.Post.CalculateSize()
   at Google.Protobuf.CodedOutputStream.ComputeMessageSize(Google.Protobuf.IMessage)
   at Google.Protobuf.FieldCodec+<>c__32`1[[System.__Canon, System.Private.CoreLib, Version=5.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e]].b__32_4(System.__Canon)

Quick links to full source:

source blog.proto
source ApplicationDbContext.cs
source BlogService.cs
source SeedData.cs

user_9855089 · Answer

I've got it working now with the code below. I don't know if this is the correct way, or if I can simplify it, I have to learn more about LINQ & mapping objects also.

I posted a question at Github Repo grpc-dotnet and the answer James Newton-King gave me actually helped me a lot: "The Protobuf serializer doesn't support reference loops." Because of that, I knew I had to modify my queries and I should not be looking for a solution to solve the reference loop in gRPC.

I've updated my working sample project at Github with this code:

(part of) /Server/Services/BlogService.cs (full source)

public override async Task GetPosts(Empty request, ServerCallContext context)
{
    var postsQuery = await dbContext.Posts.AsSplitQuery() // trying/testing ".AsSplitQuery()"
                                                          //var postsQuery = await dbContext.Posts
        .Where(ps => ps.PostStat == PostStatus.Published)
        .Include(pa => pa.PostAuthor)
        .Include(pe => pe.PostExtended)
        .Include(tipd => tipd.TagsInPostData)
        .OrderByDescending(dc => dc.DateCreated)
        .AsNoTracking().ToListAsync();

    // The Protobuf serializer doesn't support reference loops
    // see: https://github.com/grpc/grpc-dotnet/issues/1177#issuecomment-763910215
    //var posts = new Posts();
    //posts.PostsData.AddRange(allPosts); // so this doesn't work
    //return posts

    Posts posts = new();
    foreach (var p in postsQuery)
    {
        Post post = new()
        {
            PostId = p.PostId,
            Title = p.Title,
            DateCreated = p.DateCreated,
            PostStat = p.PostStat,
            PostAuthor = p.PostAuthor,
            PostExtended = p.PostExtended,
        };

        // Just add all the tags to each post, this isn't a reference loop.
        List tags = p.TagsInPostData.Select(t => new Tag { TagId = t.TagId }).ToList();
        post.TagsInPostData.AddRange(tags);

        // Add Post (now with tags) to posts
        posts.PostsData.Add(post);
    }
    return posts;
}

And this is the result:

ASP.NET Core 5 Blazor WASM, gRPC, Entity Framework Core 5: many-to-many results in stack overflow

1 Answers