Streaming an in-memory generated file in ASP.NET Core

user_313528 0
Viewed 4886

After trawling the internet for hours, I'm lost on how to solve my problem for ASP.NET Core 2.x.

I am generating a CSV on the fly (which can take several minutes) and then trying to send that back to the client. Lots of clients are timing out before I start sending a response, so I am trying to stream the file back to them (with an immediate 200 response) and write to the stream asynchronously. It seemed like this was possible with PushStreamContent previously in ASP, but I'm unsure how to structure my code so the CSV generation is done asynchronously and returning an HTTP response immediately.

[HttpGet("csv")]
public async Task<FileStreamResult> GetCSV(long id)
{
    // this stage can take 2+ mins, which obviously blocks the response
    var data = await GetData(id);
    var records = _csvGenerator.GenerateRecords(data); 

    // using the CsvHelper Nuget package
    var stream = new MemoryStream();
    var writer = new StreamWriter(stream);
    var csv = new CsvWriter(writer);

    csv.WriteRecords(stream, records);
    await writer.FlushAsync();

    return new FileStreamResult(stream, new MediaTypeHeaderValue("text/csv))
    {
        FileDownloadName = "results.csv"
    };
 }

If you make a request to this controller method, you'll get nothing until the whole CSV has finished generating and then you finally get a response, by which point most client requests have timed out.

I've tried wrapping the CSV generation code in a Task.Run() but that has not helped my issue either.

csvhelper asp.net-core .net-core asp.net-mvc c#
2 Answers

There isn't a PushStreamContext kind of type built-in to ASP.NET Core. You can, however, build your own FileCallbackResult which does the same thing. This example code should do it:

public class FileCallbackResult : FileResult
{
    private Func<Stream, ActionContext, Task> _callback;

    public FileCallbackResult(MediaTypeHeaderValue contentType, Func<Stream, ActionContext, Task> callback)
        : base(contentType?.ToString())
    {
        if (callback == null)
            throw new ArgumentNullException(nameof(callback));
        _callback = callback;
    }

    public override Task ExecuteResultAsync(ActionContext context)
    {
        if (context == null)
            throw new ArgumentNullException(nameof(context));
        var executor = new FileCallbackResultExecutor(context.HttpContext.RequestServices.GetRequiredService<ILoggerFactory>());
        return executor.ExecuteAsync(context, this);
    }

    private sealed class FileCallbackResultExecutor : FileResultExecutorBase
    {
        public FileCallbackResultExecutor(ILoggerFactory loggerFactory)
            : base(CreateLogger<FileCallbackResultExecutor>(loggerFactory))
        {
        }

        public Task ExecuteAsync(ActionContext context, FileCallbackResult result)
        {
            SetHeadersAndLog(context, result, null);
            return result._callback(context.HttpContext.Response.Body, context);
        }
    }
}

Usage:

[HttpGet("csv")]
public IActionResult GetCSV(long id)
{
  return new FileCallbackResult(new MediaTypeHeaderValue("text/csv"), async (outputStream, _) =>
  {
    var data = await GetData(id);
    var records = _csvGenerator.GenerateRecords(data); 
    var writer = new StreamWriter(outputStream);
    var csv = new CsvWriter(writer);
    csv.WriteRecords(stream, records);
    await writer.FlushAsync();
  })
  {
    FileDownloadName = "results.csv"
  };
}

Bear in mind that FileCallbackResult has the same limitations as PushStreamContext: that if an error occurs in the callback, the web server has no good way of notifying the client of that error. All you can do is propagate the exception, which will cause ASP.NET to clamp the connection shut early, so clients get a "connection unexpectedly closed" or "download aborted" error. This is because HTTP sends the error code first, in the header, before the body starts streaming.

user_2636930

If document generation takes 2+ minutes it should be asynchronous. It could be like this:

  1. client sends request to generate document
  2. you accept request, start generation in background and reply with message like generation has been started, we will notify you
  3. on client you periodically check whether document is ready and get the link finally

You also can do it with signalr. Steps are the same but it's not needed for client to check the status of document. You can push the link when document is completed.

user_10577670
Related