Large HTTPResponseMessage causes .NET Core server process to run out of memory

user_1506086 0
Viewed 636

I have a C# .NET 2.2 web server process that exposes an API. When a request comes in, the server needs to make its own HTTP request to a database API. Depending on the query, the response from the database can be very large, and in some cases this is large enough that my .NET process crashes with (Memory quota exceeded) in the logs.

The code that sends off the request looks like this:

string endpoint_url = "<database service url>";
var request_body = new StringContent(query, Encoding.UTF8, "<content type>");
request_body.Headers.ContentType.CharSet = "";
try {
    var request_task = Http.client.PostAsync(endpoint_url, request_body);
    if (await Task.WhenAny(request_task, Task.Delay(timeoutSeconds*1000)) == request_task) {
        request_task.Result.EnsureSuccessStatusCode();
        var response = await request_task.Result.Content.ReadAsStringAsync();
        JObject json_result = JObject.Parse(response);
        if (json_result["errors"] is null) {
            return json_result;
        } else {
            // return error
        }
    } else {
        // return timeout error
    }
} catch(Exception e) {
    // return error
}

My question is, what is the best way of protecting against my web service going down when a query returns a large response like this? The .NET Core best practices suggest that I shouldn't be loading the response body into a string wholesale, but doesn't really suggest an alternative.

I want to fail gracefully and return an error to the client rather than causing an outage of the .NET service, so setting some kind of limit on the response size would work. Unfortunately the database service in question does not return a Content-Length header so I can't just check that.

My web server currently has 512MB of memory available, which I know is not much, but I'm concerned that this error could happen for a large response regardless of the amount of memory I have available. My main concern is guaranteeing that my .NET service wont crash regardless of the size of response from the database service.

.net-core .net c#
3 Answers

The simplest approach that you could use is to make decision based on the returning row count.

If you are using ExecuteReader then it will not return the affected rows, but you can overcome this limitation by simply returning two result sets. The first result set would have a single row with a single column, which tells you the row count and based on that you can decide whether or not you are calling the NextResult and process the requested data.

If you are using stored procedures then you can use an out parameter to indicate the retrieved row count. By using either the @@ROWCOUNT variable or the ROWCOUNT_BIG() function. Yet again you can branch on that data.

The pro side of these solutions is that you don't have to read any record if it would outgrow your available space.

The con side of these solutions is that determining the threshold could be hard, because it could depend on the query itself, on one (or more) parameter(s) of it, on the table size, etc.

user_132688550

Well you definitely shouldn't be creating an unbounded string that could be larger than your heap size but it's more complicated than just that advice. As others are pointing out the entire system needs to work together be able to return large results with a limited memory footprint.

The simplest answer to your direct question - how can I send back an error if the response won't fit in memory - would be to create a buffer of some limited "max" size and read only that much data from the response. If it doesn't fit in your buffer then it's too large and you can return an error.

But in general that's a poor design because the "max" is impossible to statically derive - it depends on server load.

The better answer is to avoid buffering the entire result before sending it to the client and instead stream the results to the client - read in a buffer full of data and write out that buffer - or some processed form of that buffer - to the client. But that requires some synergy between the back-end API, your service and possibly the client.

If your service has to parse a complete object - as you're showing with Json.Parse - then you'll likely need to re-think your design in general.

user_14131740

If Http.client is an HttpClient you can restrict the maximum data that it will read before aborting the operation and throwing an exception with it's MaxResponseContentBufferSize property. By default it's set to 2Gb, that explains why makes your server go away if it only has 512Mb of RAM, so you can set it to something like 10/20Mb and handle the exception if it has been overflown.

user_1626990
Related