C#: How to Apply Serialization in API Controller Level to Obtain Certain Columns

Viewed 255

We currently have Repository Layer and App Service Layer.

  1. Repo gets data from SQL Server database with Entity Framework.
  2. App Service Layer Collects data, and does more things: send emails, parse flat files,

Repo Layer

public Task<Sales> GetBySalesId(int salesId)
{
    var salesData = _context.Sales
        .Include(c => c.Customer)
        .FirstOrDefaultAsync(c => c.SalesId == salesId);
    return salesData ;
}

Service Layer:

public async Task<SalesDto> GetSalesByIdAppService(int salesId)
{
    var salesData = await _salesRepository.GetBySalesId(salesId);
    var salesDto = _mapper.Map<SalesDto>(salesData);

    return salesDto;
}

This is currently working fine. However tomorrow, one of my colleagues may require More columns, when they aren't needed in my specific portion of application.

Here two more Linq includes are added: However, I do not need Product or Employee.

New Repo Addition:

public Task<Sales> GetBySalesId(int salesId)
{
    var salesData = _context.Sales
        .Include(c => c.Customer)
        .Include(c => c.ProductType)
        .Include(c => c.Employee)
        .FirstOrDefaultAsync(c => c.SalesId == salesId);
    return salesData ;
}

Background One Suggestion is create Another Middle Domain Layer that everyone can utilize. In the API DTO Level, everyone can have separate DTOs, which only collects the only required class members from Domain. This would essentially entail creating Another layer, where DTO is a subset of the new "Domain" layer.

*Another suggestion, is to apply serialization only for the columns which are required. I keep hearing about this, however, how can this be done? Is it possible to application serialization to the Controller API without adding another layer? Does Newtonsoft have a tool, or any syntax in C# ?

API Controller

public async Task<ActionResult<SalesDto>> GetSalesBySalesId(string salesId)
{
    var dto = await _service.GetBySalesId(salesId);
    return Ok(dto);
}

JSON Ignore may not work , because we all share same DTO, and ignoring for one area, may be required for other part of application.

4 Answers

Decorate your members in class with [JsonIgnore] attribute which are NOT required in response. JsonIgnore is available in System.Text.Json.Serialization namespace.

 public class SalesDto
{

    [JsonIgnore]
    public string Customer { get; set; }
    public string ProductType { get; set; }

    public string Employee { get; set; }
}

Bind all properties of model with data and when you send it to UI, Customer property would not be available in response.

We should fetch all the data from database and the process that data in our presentation layer. GraphQL, could be a winner for this scenario but need to explore

OData as a Middle Domain Layer can be useful to support this type of requirement. It gives the caller some control over the shape of the Object Graph that is returned. The code to acheive this is too involved to include as a single POST, however these types of requirements are often better solved through implementing an architecture that is designed specifically to deal with them, rather than rolling your own quick fix.

  • You can also look at GraphQL that allows greater flexibility over the shape of the returned data, however that is a steeper learning curve for the server and client side.

The problem with just using JsonIgnore is that this is a permanent definition on your DTO, which would make it hard to use the same DTO definitions for different applications that might require a different shape/view of the data.

A solution to that problem is then to create a tightly-coupled branch of DTOs for each application that inherits from the base but overrides the properties by decorating their properties with JsonIgnore attributes.

You want to avoid tightly coupled scenarios where the UI enforces too much structure on your data model, this can will make it harder to maintain your data model and can lead to many other anti-patterns down the track.


OData allows a single set of DTOs that have a consistent structure in the backend and the client end, whilst allowing the client end to reduce/omit/ignore fields that it either does not know about yet or does not want transmitted.

The key here is that the Client now has (some) control over the graph, rather than the API needing to anticipate or tightly define the specific properties that each application MUST consume.

It has a rich standard convention based query language that means many off the shelf products may be able to integrate directly with your API

Some Pros for consideration:

  1. Single Defintion of DTOs

    • Write once, defining the full structure of your data model that all applications will have access to.
    • Clients can specify the fields that they need through query projections.
  2. Expose Business Logic through the API

    • This is a feature of all APIs / Service layers, however OData has a standard convention for supporting and documenting custom business Functions and Actions
  3. Minimal DTOs over the wire

    • OData serializer can be configured to only transmit properties that have values across the wire
    • Callers can specify which fields to include in query results
    • You can configure the default properties and navigation links to send for resource requests when the client does not specify the projection.
  4. PATCH (delta) Updates

    • OData has a simple mechanism to support partial updates to objects, you only send the changed properties over the wire for any object
      • Sure, this is really part of the previous point, but it is a really strong argument for using OData over a standard REST based API
    • You can easily inject business rules into the query validation pipeline (or the execution) to prevent updates to specific fields, this can include role based or user based security context rules.
  5. .Net LINQ Support
    Whilst the API is accessed via URL conventions, these conventions can be easily mapped to .Net LINQ queries on the client side via the ODataLib or Simple OData Client.

    • Not all possible LINQ functions and queries are supported, but there is enough there to get the job done.
    • This means you can easily support C# client side applications.
  6. Versioning can be avoided
    This becomes a Con as well, however additive changes to the data model can be easily absorbed into the runtime without having to publish a new version of the API.

    All of the above points contribute to allow Additive changes to be freely supported by down level clients without interruptions.

    • It is possible to also map some old fields across to new definitions in cases where you want to rename fields, or you can provide default implementations for fields that have been removed.
      • The OData Model is a configuration that defines how URL requests are mapped or translated to endpoints on the API controllers, meaning there is a layer where you can change how client requests are mapped to

Key Cons to be aware of:

  1. Performance - OData API uses HTTP protocols, so there is an inherent performance hit when compared to local DLL calls, windows services or RPC, even when the API and Application are located on the same machine

    • This performance hit can be minimised and is usually an acceptable overall cost
    • GraphQL and most other REST/HTTP based API architectures will have similar JSON over HTTP performance issues.

    OData is still a Good fit for JSON over HTTP based remote API hosting scenarios, like javascript based clients.

  2. Poor support for updating related objects
    While PATCH is great for single objects, it doesn't work for nested object graphs, to support updates to nested objects you either need to manually keep a repository of changes on the client side and manually call path on each of the nested objects.

    • The OData protocol does have a provision for batching multiple queries so they can be executed as an atomic operation, but you still have to construct the individual updates.
    • There is a .Net based OData Client Generation Tool can be used to generate a client side repo to manage this for you.

    How often are you expecting your client to send back a rich collection of objects to update in a single hit?

    Is it a good idea to allow the client to do this, does an omitted field from the client mean we should set that property to null, does it mean we should delete that related data?

    Consider creating actions on the API to execute operations that affect multiple records to keep your clients thin and to consolidate logic so that each of your client applications does not have to re-implement the complex logic.

  3. Version Support
    Versioning can be a long term issue if you want to allow destructive changes to your data model and DTOs. While the standard URL conventions support versioning the code to implement it is still complex

    Versioning can be hard to implement in any APIs, however the ability to set default projections for each DTO and with the client able to control their own specific projections means that the OData model is more resilient to additive-only changes, such as adding more tables, or fields.

    Additive changes can be implemented without interrupting client applications.

  4. Fixed Schema
    Although clients can request specific fields to be sent over the wire for the DTOs, including navigation properties, the client cannot easily request the data to come back in a totally different structure. Clients can only really request that certain fields are omitted from results.

    • There is support for $apply URL parameter to return the results of aggregate operations, but this doesn't provide full control over the shape of the results.

    GraphQL does address this exact issue. Moving the mapping from the API to the client side, giving the client a more control over the schema.

the solution for this problem is very simple , you can use half generic query for this , WITHOUT changing anything in your DTO.

first let your repo function takes a generic include like this:

public Task<Sales> GetBySalesId(string salesId, Func<IQueryable<Sales>, IIncludableQueryable<Sales, object>> include = null)
    {
        var query = _context.Sales.Where(x => x.Id == salesId);
        if (include != null)
            query = include(query);
        var salesData = query.FirstOrDefaultAsync();
        return salesData;
    }

this can be used in service layer like this:

public async Task<Sales> GetById(string salesId)
    {
        var result = await _yourRepo.GetBySalesId(salesId, 
            include:  source => source
                .Include(a => a.SOMETHING)
                .Include(a => a.SOMETHING)
                .ThenInclude(a => a.SOMETHING));
        return result;
    }

now to specialize the query result you can either do it based on your token (if you are using authorization in your api) or can create a number of service functions , call them based on a condition you recieve in the controller like an integer or something.

public async Task<Sales> test(string salesId)
    {
        Func<IQueryable<Sales>, IIncludableQueryable<Sales, object>> include = null;
        if (UserRole == "YOU")
            include = a => a.Include(a => a.SOMETHING);
        else if (UserRole == "SomeoneElse")
            include = a => a.Include(a => a.SOMETHING).ThenInclude(a=>a.SOMETHINGELSE);

        var result = await _yourRepo.GetBySalesId(salesId, 
            include: include);
        return result;
    }

First of all

your logic is strange: you request DB return all columns and then take only few needed, it is inefficient. Imagine you have 20 columns...

var salesData = await _salesRepository.GetBySalesId(salesId);
var salesDto = _mapper.Map<SalesDto>(salesData);

Should a sales repository be able to include customers and products?

It is probably a holywar subject. Generally speaking, if your architecture wont allow you to switch from DB to file storage and from ASP.NET MVC to Console App, most likely it has design flaws (and it might be perfectly OK for your company current needs)

Summary

You need to make more service methods that do build results not just transferring data from Repo to caller as is. For your case you need your service to cover more scenarios

  • AppService.GetSalesById(salesId)
  • AppService.GetSalesWithProductsById(sales)
  • AppService.GetSalesById(salesId, includeProducts, includeCustomers)

My personal preference is to change multiple parameters with Commands and make service methods return Results.

If your colleague were to add say 2 columns - it is easier to add them to existing result, if colleague is writing something new – its better to introduce new Method and result

Commands and results

Command stands for some situation and its variations, service looks nice are clean. This approach is time tested on one of my projects for last 10 years. We have switched database 3 times, several ORMs and 2 UIs. To be specific we use ICommand and IResult to make it super flexible.

API Controller

public async Task<ActionResult<SalesDto>> GetSalesBySalesId(string salesId)
{
    UISalesTable dto = await (UISalesTable) _service.GetSales(new GetSalesForUICommand
    {
         SalesId = salesId,
         IncludeProductType = true,
         IncludeCustomer = false
    });

    return Ok(dto);
}

public async Task<ActionResult<MonthlySales>> GetSalesReport(string salesId)
{
    MonthlySales dto = await (MonthlySales) _service.GetSales(new GetMonthlySalesReportCommand
    {
         SalesId = salesId,
         // extra filters goes here
    });

    return Ok(dto);
}

Service layer You make as many DTOs as there are results (it costs nothing)

public async Task<UISalesTable> GetSales(GetMonthlySalesReportCommand command)
{
    UISalesTable result = new UISalesTable();

    // good place to use Builder pattern
    result.SalesByMonthes = .....;

    TopProductsCalculator calc = new TopProductsCalculator();
    result.TopProducts = calc.Calculate(command.FromDate, command.ToDate);

    result.Etc = .....;

    return result;
}

It all depends

Unfortunately, there is no recipe. It is always tradeoff between quality and time-to-market. Last years I preferer to keep things simple and even abandoned idea of repositories and now I work with DataContext directly because if I were to switch say to MongoDB, I would have to write every single repository method again and it happens few times in a life.

Related