Stream a multi-GB file to AWS S3 from ASP.NET Core Web API

Viewed 3516

I wish to create a large (multi-GB) file in an AWS S3 bucket from an ASP.NET Core Web API. The file is sufficiently large that I wish not to load the Stream into memory prior to uploading it to AWS S3.

Using PutObjectAsync() I'm forced to pre-populate the Stream prior to passing it on to the AWS SDK, illustrated below:

var putObjectRequest = new PutObjectRequest
{
    BucketName = "my-s3-bucket",
    Key = "my-file-name.txt",
    InputStream = stream
};
var putObjectResponse = await amazonS3Client.PutObjectAsync(putObjectRequest);

My ideal pattern would involve the AWS SDK returning a StreamWriter (of sorts) I could Write() to many times and then Finalise() when I'm done.

Two questions concerning my challenge:

  • Am I misinformed about having to pre-populate the Stream prior to calling on PutObjectAsync()?
  • How should I go about uploading my large (multi-GB) file?
1 Answers
Related