Reduce the size of the node-modules zip sent by the serverless framework

user_51197 0
Viewed 9380

I am writing a few nodejs functions using the serverless framework. The package.json file requires a few dependencies:

{
  "name": "adam-test-sls",
  "version": "0.1.0",
  "description": "Test package to play with sls/lambda",
  "main": "handler.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "Adam Matan <adam@binaris.com>",
  "license": "UNLICENSED",
  "dependencies": {
    "aws-sdk": ">= 2.0.9",
    "json": "^9.0.6",
    "underscore": "^1.8.3",
    "uuid": "^3.1.0"
  },
  "devDependencies": {
    "eslint": "^4.2.0",
    "eslint-config-airbnb": "^15.0.2",
    "eslint-config-airbnb-base": "^11.2.0",
    "eslint-plugin-import": "^2.7.0"
  }
}

The size of the node-modules directory is almost 50mb:

# du -smc node_modules
47  node_modules
47  total

Deployment time is more than 35 seconds, given a zip size of ~9.5MB:

# time serverless deploy function --function hello -v
Serverless: Packaging function: hello...
Serverless: Uploading function: hello (9.46 MB)...
Serverless: Successfully deployed function: hello
serverless deploy function --function hello -v  4.28s user 1.15s system 15% cpu 35.165 total

This is somewhat inefficient - I only change one file, but I have to pack all the unchanged dependencies whenever I make the slightest change.

Any idea how to reduce the zip size (perhaps removing devDependencies), or upload only the changed file?

aws-lambda npm serverless-framework node.js dependencies
4 Answers

I used layers for this: Layers are used to pull in additional code and content in the form of layers. Which will keep your lambda functions small and will pull dependencies from layers in runtime.

Step 1: Create a folder somewhere called nodejs

cd Desktop
mkdir nodejs
cd nodejs
npm init

Step 2: In this folder install your dependencies which are large in size. For example:

npm install --save geoip-lite

Step 3: Zip it, give name of your choice.

Step 4: Login to aws console, go to lambda service and then choose create layer. Then upload zip file you created. (As it says you can upload this zip file to S3 in case its larger than 10 mb)

At this step you can assign layer to lambda by going into lambda function and add custom layer which you created from above steps.

In case you are interested to add layer is yml config

functions:
  hello:
   handler: handler.hello
   layers:
    - arn:aws:lambda:region:XXXXXX:layer:LayerName:Y
user_16144280

To solve this problem I used a plugin called serverless-plugin-scripts and wrote a shell script that runs on after:package:createDeploymentArtifacts hook.

The script unpacks deployment package, removes node_modules directory, runs npm install --only=prod and packs everything back. For some reason npm prune --production did not work well for me at all.

This is the configration (serverless.yml):

service: my-service-name
provider:
  name: aws
  runtime: nodejs8.10
  region: us-east-1
  environment:
    NODE_ENV: ${self:custom.stage}

custom:
  stage: ${opt:stage, self:provider.stage}
  region: ${opt:region, self:provider.region}
  scripts:
    hooks:
      'after:package:createDeploymentArtifacts': >
        printf "[after:package:createDeploymentArtifacts hook] Removing development dependencies " &&
        (cd .serverless && unzip -qq my-service-name.zip -d my-service-name)          && printf "." &&
        (cd .serverless && rm -rf my-service-name/node_modules)                       && printf "." &&
        (cd .serverless/my-service-name && npm install --only=prod > /dev/null 2>&1)  && printf "." &&
        (cd .serverless/my-service-name/ && zip -q -FSr ../my-service-name.zip .)     && printf ".\n" &&
        rm -rf .serverless/my-service-name/ &&
        printf "[after:package:createDeploymentArtifacts hook] Done\n"

functions:  
  scheduledHandler:
    handler: src/handlers.scheduledHandler
    events:
      - schedule: rate(10 minutes)

package:
  exclude:
  - .vscode/**
  - .idea/**
  - test/**
  - build/**
  - .build/**

plugins:
  - serverless-plugin-scripts
user_10325320
Related