I am trying to find out the optimal way to upload a large file to the server. On my local, I tried both methods:
- Sequential Chunked Upload, where file is broken down into multiple chunks and uploaded one at a time to server. On the server side, the incoming chunk is directly appended to the final file.
- Parallel Chunked Upload, where the chunks are uploaded in parallel to the server. The server stores the files as temp small files, and then merges them into one complete file when all the chunks are uploaded.
But here the merging time is way more as compared to upload time, atleast on my local. Hence, the parallel upload is always a lot slower than the sequential upload. How can I improve my upload time?
Here is the Sequential Upload Code :
[HttpPost]
[RequestFormLimits(MultipartBodyLengthLimit = 209715200)]
[RequestSizeLimit(209715200)]
public async Task<IActionResult> UploadSeq()
{
string filename = "./Uploads/seqfile";
using(Stream file = Request.Body)
using (FileStream stream = new FileStream(filename, FileMode.Append))
{
await file.CopyToAsync(stream);
await stream.FlushAsync();
}
return Ok();
}
And here is the parallel code :
[HttpPost("/ParUpload/{id}/{size}")]
[RequestFormLimits(MultipartBodyLengthLimit = 209715200)]
[RequestSizeLimit(209715200)]
public async Task<IActionResult> UploadParallel(int id, int size)
{
string filename = "./Uploads/parfilecomplete.mkv";
using (Stream file = Request.Body)
using (FileStream stream = new FileStream(filename, FileMode.OpenOrCreate, FileAccess.Write, FileShare.Write))
{
stream.Seek(id * size, SeekOrigin.Begin);
await file.CopyToAsync(stream);
await stream.FlushAsync();
}
return Ok();
}