I'm writing a Lambda function to upload an array of images to S3, with each image, I will need to upload the original one and generate a thumb for it (I'm using sharp). When I test it, sometimes it works, sometimes it doesn't, images can't be uploaded. I tried to increase RAM but it doesn't work well and the there are logs from CloudWatch to trace.
I don't know why, can you give me best solution/refactor idea for this code. I have code like this:
index.js, get array of images as param, then run the loop and call image upload function.
import { resizeAndUpload } from './aws.js'
export async function handler(event, context) {
const { imgs, filename} = event
try {
for (const [i, img] of imgs.entries()) {
const upload = await resizeAndUpload(img, `${filename}_${i}`)
console.log('upload/result', upload.img, upload.thumb)
// upload && context.succeed()
}
}
catch (err) {
context.fail(`Error resizing img: ${err}`)
}
}
aws.js, the main upload, resize image function. On this function, I combine 2 uploads, one for original img, one for thumb.
import aws from 'aws-sdk'
import fetch from 'node-fetch'
import sharp from 'sharp'
// AWS config
aws.config.update({
secretAccessKey: process.env.AMZ_S3_SECRET_ACCESS_KEY,
accessKeyId: process.env.AMZ_S3_ACCESS_KEY_ID,
region: process.env.AMZ_S3_REGION,
})
export const s3 = new aws.S3()
const imgWidth = 500
// resize & upload function
export const resizeAndUpload = async (imgUrl, filename) => {
return await fetch(imgUrl)
.then(async res => {
const arrayBuffer = await res.arrayBuffer()
const imgBuffer = Buffer.from(arrayBuffer)
const resizedImg = await sharp(imgBuffer).resize(imgWidth, null).toBuffer()
// thumb
const uploadThumb = s3.upload({
Bucket: process.env.AMZ_S3_BUCKET,
Key: `public/gallery_${filename}_thumb.webp`,
cacheControl: 'max-age=604800',
ContentType: res.headers['content-type'],
ContentLength: res.headers['content-length'],
Body: resizedImg
}).promise()
// main img
const uploadImg = s3.upload({
Bucket: process.env.AMZ_S3_BUCKET,
Key: `public/gallery_${filename}.webp`,
cacheControl: 'max-age=604800',
ContentType: res.headers['content-type'],
ContentLength: res.headers['content-length'],
Body: imgBuffer
}).promise()
const result = {
img: (await uploadImg).Location,
thumb: (await uploadThumb).Location
}
return result
})
.catch(err => {
console.log('upload err', err)
return null
})
}
Thank you for you support