How to upload a large file through an Azure function?

Viewed 22019

I am exploring Azure Functions. The scenarios I have tested so far work great.

I am at a point where I am trying to figure out a way to upload files (20MB+) through an Azure Function.

The idea is that the Azure Function would first validate whether or not the authenticated user is allowed to upload the file before getting a hold on the request's stream and saving it to the BLOB storage.

Here is the code from the client side which creates a StreamContent to beam the bytes to the server:

using (Stream fileStream = ...)
{
    var streamContent = new StreamContent(fileStream);

    streamContent.Headers.ContentType = new MediaTypeHeaderValue("application/octet-stream");
    streamContent.Headers.ContentLength = fileStream.Length;
    streamContent.Headers.Add("FileId", fileId);

    var responseMessage = await m_httpClient.PutAsync(<validURI>, streamContent);

    responseMessage.EnsureSuccessStatusCode();

    succeeded = true;
}

Here is the code on the server side.

[FunctionName("upload-data")]
public static async Task<HttpResponseMessage> Run([HttpTrigger(AuthorizationLevel.Function, "put")]HttpRequestMessage req, TraceWriter log)
{
    try
    {
         //  Initialize stuff.

         //  Validate authenticated user & privileges.  

         //  Get the content stream of the request and 
         //  save it in the BLOB storage.

         return req.CreateResponse(HttpStatusCode.OK);
    }
    catch (Exception exc)
    {
        return req.CreateResponse(HttpStatusCode.InternalServerError, exc);
    }
}

I put a breakpoint right at the beginning of the method. I was expecting the breakpoint to be hit right after the client side sent the request, no matter how big the file is. However it does not.

I am guessing that the Azure Function is somehow trying to get all the content of the request's body before calling the method. I also think that I am sending a file that may exceed the 4 MB limit of the underlying Web Job but I did not see a way to configure that.

Is it possible to upload a large file to an Azure Function by streaming it? Is there a way to make this work?

4 Answers

An alternative solution (maybe not best practice) is POSTing the file using chunks. But it requires you to keep track of these chunks and consolidating them afterwards.

For example:

  • Front End - Send multiple chunks to endpoint (multiple POSTs)
  • Back End - Stores chunks
  • Front End - Once sending is complete, confirm with endpoint
  • Back End - Merges chunks

Dropzone JS - Chunking

Related