Hangfire jobs insertion impacted on the API performance of the application

user_4753489 0
Viewed 981

I'm Working with ASP.NET Core 2.2 Web API multi-tenant application. The application uses Hangfire to run background tasks. We are trying to improve the performance of the application.

It stores all the jobs in a separate DB (Hangfire DB), but this impact on API performance. I have traced the API request in order to check the request time, here is the result:

API Result

Here is the code

public async Task<string> AddUser(UserModel user)
{
    CreateUserInBackgorund(user);

    // removed code

    return "some status";
}

[Queue(Constants.Critical)]
public void CreateUserInBackgorund(UserModel user)
{
    BackgroundJob.Enqueue(() => CreateUser(user));
}

public async Task CreateUser(UserModel user)
{
    try
    {
        //Other code
    }
    catch (Exception ex)
    {
        _logger.Error(ex.Message, ex);
    }
}

The trace logs seem that background calls impacted on the performance of the request. Is there a way to reduce this time or use another approach?

hangfire asp.net-core asp.net-core-webapi performance c#
1 Answers

Because of some lack of information, here are suggestions on what to check:

  1. There are issues might be issues with SQL Server THREADPOOL Waits rather than with C# code. https://www.sqlskills.com/help/waits/threadpool/ Also, you may get more information about long running queries by check SQL Server tools https://www.sqlshack.com/how-to-identify-slow-running-queries-in-sql-server/ Please, be aware each transaction locks the table, and if there are a lot of transactions in queue they may significantly reduce performance as well.

  2. In [Queue(Constants.Critical)] there is definitely logic that the main request thread needs to execute before to start execute method. It could be also reasonable to try something like Task.Run(()=> CreateUserInBackgorund(user)); in AddUser(UserModel user) and to check how it would influence.

  3. It's weird practice to use background jobs in such way. Usually background jobs are utilized as schedulers for the business logic parts that are not executed by users and for those cases when some logic needs to be executed in specific time or period. E.g. calculate something each day, or each week, month and etc. If you are using background jobs just only for the purpose to pass response to the user faster, it would be better to use some Queue / Service Bus e.g. RabbitMQ for messaging events and than to add listeners that would be executing the required tasks on_message_received event on demand. So it would not be necessary to use hangfire for those things that it has not been really designed and for those massive amount of user requests. It would gain possibility to significantly reduce the amount of hangfire related requests to its database and the overall pressure on SQL Server as well.

user_130548810
Related