I wish to create a large (multi-GB) file in an AWS S3 bucket from an ASP.NET Core Web API. The file is sufficiently large that I wish not to load the Stream into memory prior to uploading it to AWS S3.
Using PutObjectAsync() I'm forced to pre-populate the Stream prior to passing it on to the AWS SDK, illustrated below:
var putObjectRequest = new PutObjectRequest
{
BucketName = "my-s3-bucket",
Key = "my-file-name.txt",
InputStream = stream
};
var putObjectResponse = await amazonS3Client.PutObjectAsync(putObjectRequest);
My ideal pattern would involve the AWS SDK returning a StreamWriter (of sorts) I could Write() to many times and then Finalise() when I'm done.
Two questions concerning my challenge:
- Am I misinformed about having to pre-populate the
Streamprior to calling onPutObjectAsync()? - How should I go about uploading my large (multi-GB) file?