PROBLEM

I am attempting to get multiple files from the AWS S3 bucket. I am using the AWS SDK v3, which makes use of the GetObjectCommand() API. This command returns a ReadableStream which I am piping into the res object so a single image loads successfully when I make a 'GET' request to my single image endpoint. However, when I create a loop and make multiple calls and attempt to pipe the data to res, the whole thing crashes and throws the error:

events.js:292
      throw er; // Unhandled 'error' event
      ^

Error [ERR_STREAM_WRITE_AFTER_END]: write after end
    at writeAfterEnd (_http_outgoing.js:668:15)
    at write_ (_http_outgoing.js:680:5)
    at ServerResponse.write (_http_outgoing.js:661:15)
    at IncomingMessage.ondata (internal/streams/readable.js:719:22)
    at IncomingMessage.emit (events.js:315:20)
    at IncomingMessage.EventEmitter.emit (domain.js:467:12)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at IncomingMessage.Readable.push (internal/streams/readable.js:223:10)
    at HTTPParser.parserOnBody (_http_common.js:139:24)
    at TLSSocket.socketOnData (_http_client.js:509:22)
    at TLSSocket.emit (events.js:315:20)
    at TLSSocket.EventEmitter.emit (domain.js:467:12)
    at addChunk (internal/streams/readable.js:309:12)
    at readableAddChunk (internal/streams/readable.js:284:9)
    at TLSSocket.Readable.push (internal/streams/readable.js:223:10)
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:188:23)
Emitted 'error' event on ServerResponse instance at:
    at ServerResponse.onerror (internal/streams/readable.js:760:14)
    at ServerResponse.emit (events.js:315:20)
    at ServerResponse.EventEmitter.emit (domain.js:467:12)
    at writeAfterEndNT (_http_outgoing.js:727:7)
    at processTicksAndRejections (internal/process/task_queues.js:81:21) {
  code: 'ERR_STREAM_WRITE_AFTER_END'
}

WHAT I HAVE TRIED

Since I am using async/await, I tried to have an async callback in the forEach loop which makes a call for every single key in the imageKeys array I send in the 'GET' request, but this doesn't work. I'm not sure how to solve this issue of being able to pipe the multiple data streams to res without it closing. Would be great to get some help! NOTE: getFile() is a custom function imported in filesController.js.

EXPRESS CONTROLLERS (filesController.js)

For single file (working):

files.get("/image/:key", async (req, res) => {
  try {
    const key = req.params.key;
    // data returned is a ReadableStream. check out Node.js documentation for Readable Streams
    const data = await getFile(carfixBucket, key);
    // a readable stream can pipe data to a writable stream 'res'. 
    // I.E you READ (copy data from) a readable stream and you WRITE (copy data to) a writable stream. 
    // a 
    data.pipe(res);
    // res.status(200).send(data)
  } catch (err) {
    res.status(500).send(err);
  }
});

For multiple files(not working):

files.get("/images", async (req, res) => {
  try {
    const { imageKeys } = req.body; 
    console.log(imageKeys)
    imageKeys.forEach( async (key) => {
      const data = await getFile(carfixBucket, key)
      data.pipe(res)    
    })
    // res.status(200).send("Success")
  } catch (err) {
    res.status(500).send(err);
  }
})

Custom Function

getFile is a function I created and exported which accepts the name of the S3 bucket and the key(name) of the file stored in the bucket. It returns a ReadableStream.

const getFile = async (bucketName, key) => {
  const bucketParams = {
    Bucket: bucketName,
    Key: key,
  };
  try {
    // the response returned from Bucket contains a ReadableStream on 'Body' field. to convert to a string if necessary
    const response = await S3.send(new GetObjectCommand(bucketParams));
    return response.Body;
  } catch (err) {
    return err;
  }
};