How to save file from S3 using aws-sdk v3

user_19217 0
Viewed 4320

With the new aws-sdk v3, specifically @aws-sdk/client-s3, it's not clear to me how to store files to disk from S3.
Having the following code

const { Body } = await s3.send(new GetObjectCommand({
  Bucket: 'myBucket',
  Key: 'myKey',
}))

The Body can be of type Readable | ReadableStream | Blob.
I'd like to store this into a file on disk. Since the return types are completely different, is there an easy way to e.g. get a Buffer or Stream.Readable from aws-sdk?

In aws-sdk v2 it was easy with .createReadStream() function which seems to no longer exist.

aws-sdk-nodejs aws-sdk node.js amazon-s3
3 Answers

It appears ReadableStream | Blob is available only in browser. Node.js always gets Readable which is what I need.

The solution is pretty straightforward afterwards with fs.createWriteStream and piping the Readable such as

await new Promise((resolve, reject) => {
  Body.pipe(fs.createWriteStream(filePath))
    .on('error', err => reject(err))
    .on('close', () => resolve())
})

For reference, here's an issue where improvement to documentation is discussed https://github.com/aws/aws-sdk-js-v3/issues/1877

user_192170

GetObjectCommand returns readable stream, we use fs module to create writeStream and pass the returned readableStream from GetObjectCommand to writeStream to save the file 

const fs = require('fs');
const { S3Client, GetObjectCommand } = require('@aws-sdk/client-s3');

const s3Client = new S3Client({
  region: 'your_region',
  credentials: { accessKeyId: 'your_accessKeyId', secretAccessKey: 'your_secretAccessKey' },
});

const bucketParams = {
  Bucket: 'your_bucket',
  Key: 'your_key.jpg',
};

const run = async () => {
  try {
    const data = await s3Client.send(new GetObjectCommand(bucketParams));
    const inputStream = data.Body;
    const downloadPath = 'example.png';
    const outputStream = fs.createWriteStream(downloadPath);
    inputStream.pipe(outputStream);
    outputStream.on('finish', () => {
      console.log(`downloaded the file successfully`);
    });
  } catch (err) {
    console.log('Error', err);
  }
};

user_123823910

To do this in typescript you can also do

await new Promise((resolve, reject) => {
 if (Body instanceof Readable) {
   Body.pipe(fs.createWriteStream(filePath))
    .on('error', err => reject(err))
    .on('close', () => resolve())
 }
});
user_76965640
Related