AWS Lambda function to upload and generate thumb sometimes not work

Viewed 23

I'm writing a Lambda function to upload an array of images to S3, with each image, I will need to upload the original one and generate a thumb for it (I'm using sharp). When I test it, sometimes it works, sometimes it doesn't, images can't be uploaded. I tried to increase RAM but it doesn't work well and the there are logs from CloudWatch to trace.

I don't know why, can you give me best solution/refactor idea for this code. I have code like this:

index.js, get array of images as param, then run the loop and call image upload function.

import { resizeAndUpload } from './aws.js'

export async function handler(event, context) {
    const { imgs, filename} = event
    try {
        for (const [i, img] of imgs.entries()) {
            const upload = await resizeAndUpload(img, `${filename}_${i}`)
            console.log('upload/result', upload.img, upload.thumb)
            // upload && context.succeed()
        }
    }
    catch (err) {
        context.fail(`Error resizing img: ${err}`)
    }
}

aws.js, the main upload, resize image function. On this function, I combine 2 uploads, one for original img, one for thumb.

import aws from 'aws-sdk'
import fetch from 'node-fetch'
import sharp from 'sharp'

// AWS config
aws.config.update({
    secretAccessKey: process.env.AMZ_S3_SECRET_ACCESS_KEY,
    accessKeyId: process.env.AMZ_S3_ACCESS_KEY_ID,
    region: process.env.AMZ_S3_REGION,
})

export const s3 = new aws.S3()
const imgWidth = 500

// resize & upload function
export const resizeAndUpload = async (imgUrl, filename) => {
    return await fetch(imgUrl)
        .then(async res => {
            const arrayBuffer = await res.arrayBuffer()
            const imgBuffer = Buffer.from(arrayBuffer)
            const resizedImg = await sharp(imgBuffer).resize(imgWidth, null).toBuffer()

            // thumb
            const uploadThumb = s3.upload({
                Bucket: process.env.AMZ_S3_BUCKET,
                Key: `public/gallery_${filename}_thumb.webp`,
                cacheControl: 'max-age=604800',
                ContentType: res.headers['content-type'],
                ContentLength: res.headers['content-length'],
                Body: resizedImg
            }).promise()

            // main img
            const uploadImg = s3.upload({
                Bucket: process.env.AMZ_S3_BUCKET,
                Key: `public/gallery_${filename}.webp`,
                cacheControl: 'max-age=604800',
                ContentType: res.headers['content-type'],
                ContentLength: res.headers['content-length'],
                Body: imgBuffer
            }).promise()

            const result = {
                img: (await uploadImg).Location,
                thumb: (await uploadThumb).Location
            }
            return result
        })
        .catch(err => {
            console.log('upload err', err)
            return null
        })
}

Thank you for you support

0 Answers
Related