Unzipper for Amazon S3 using Lambda and Node.JS sets content-type as application/octet-stream, and last file in the zip is not extracted

Viewed 940

I have successfully integrated unzipper inside my Lambda. Here's the code:

console.log('Loading function');

const aws = require('aws-sdk');

const s3 = new aws.S3({ apiVersion: '2006-03-01' });

const unzipper = require("unzipper");

exports.handler = async (event) => {
    const bucket = 'my-bucket';
    const filename = decodeURIComponent(event.Records[0].s3.object.key.replace(/\+/g, ' '));
    const filepath = filename.substring(0, filename.lastIndexOf("/") + 1);

    console.log("Filename: " + filename);
    console.log("Filepath: " + filepath);

    const params = {
        Key: filename,
        Bucket: bucket
    };

    const zip = s3
        .getObject(params)
        .createReadStream()
        .pipe(unzipper.Parse({ forceStream: true }));

    const promises = [];

    let num = 0;

    for await (const e of zip) {
        const entry = e;
        const fileName = entry.path;
        const type = entry.type;

        if (type === 'File') {
            const uploadParams = {
                Bucket: bucket,
                Key: filepath + fileName,
                Body: entry,
            };

            promises.push(s3.upload(uploadParams).promise());
            num++;
        } else {
            entry.autodrain();
        }
    }

    await Promise.all(promises);
};

The problem here is that all the files extracted are set as application/octet-stream for its content-type.wrong metadata

I have confirmed that only the zip files extracted would end up with the said content-type. All of my other file uploads ended up being set to their proper content-type.

In the line of code const entry = e;, I assume that all of the objects inside the extracted zip file would contain all of the data from its previous zip folder, including each of its associated metadata.

The second problem here is that the last file in the zip folder is not extracted.

Here is the directory of my zip file:

zip_folder.zip
    | - files               // a folder. extracted with wrong content-type
    | - js                  // a folder. extracted with wrong content-type
    | - mobile              // a folder. extracted with wrong content-type
    | - book.swf            // a file. extracted with wrong content-type
    | - bookContent.swf     // a file. extracted with wrong content-type
    | - index.html          // a file. extracted with wrong content-type
    | - shot.png            // a file. not extracted at all

But upon checking my S3, the shot.png is not extracted at all. shot.png not extracted

1 Answers

I have fixed the issue. Here's the full link on how I fixed it..

Basically, you need to upload the content-type for each entry being uploaded, and S3/Lambda does not attach each corresponding content-type for each entry.

Related