Invoking second async function in lambda not invoking

user_3307520 0
Viewed 763

I am using serverless to deploy aws lambdas. Currently my set up is that api gateway triggers one lambda then I async invoke another lambda.

serverless.yml

service: scraper
useDotenv: true #added this because deprecation notice
frameworkVersion: '2'

provider:
  name: aws
  runtime: nodejs14.x
  lambdaHashingVersion: 20201021
  stage: prod
  region: us-east-2

functions:
  scraperRunner:
    handler: handler.scraperRunner
    provisionedConcurrency: 0
    timeout: 90
    events:
      - http:
          path: process/run
          method: post
          async: true
  runTargetSite:
    handler: handler.runTargetSite

plugins:
 - serverless-offline

then I have a handler.js file where I define both handler functions:

'use strict';

const startScraper = require('./crawler/runner');
const AWS = require('aws-sdk');
AWS.config.region = 'us-east-2';
const lambda = new AWS.Lambda();

function processResponse(msg, statusCode, event) {
    return {
        statusCode: statusCode,
        headers: {
            'Access-Control-Allow-Origin': '*',
            'Access-Control-Allow-Headers': 'Content-Type',
            "Access-Control-Allow-Methods": "OPTIONS,POST,GET"
        },
        body: JSON.stringify({
            message: msg,
            input: event
        })
    }
}


module.exports.scraperRunner = async (event) => {
    if (event) {
        const data = event;
        if (data.hasOwnProperty('target_id')) {

            const prov = [<array of objects>]

            for (const p of prov) {
                data['target'] = p.name
                const params = {
                    FunctionName: 'provider-scraper-prod-runTargetSite',
                    InvocationType: 'Event',
                    // LogType: 'Tail',
                    Payload: JSON.stringify(data)
                };

                await lambda.invoke(params).promise()
            }
            return processResponse(
                'Event successfully received',
                200, event)

        }
    }

    return processResponse(
        'Invalid body payload',
        402, event)
};

module.exports.runTargetSite = async function(event, context) {
    await startScraper(event)
    return processResponse(
        'Event successfully received for runTargetSite',
        200, event)
}

enter image description here

I've also added the necessary roles (I think):

  • AWSLambdaExecute
  • AWSLambdaBasicExecutionRole
  • AWSLambdaRole
  • InvokeFunction
  • InvokeAsync

Using the command line, I'm able to call the first lambda which then calls the second:

serverless invoke local --function scraperRunner --data '{"target": "_tpp_"}'

However when submitting a post request to the API Gateway endpoint, the first lambda (scraperRunner) triggers but the second one is not invoked.

aws-sdk-js aws-lambda node.js
1 Answers

When you are using the serverless invoke local command, you are invoking the lambda function directly with event defined as '{"target": "tpp"}'. When you are invoking the Lambda function through API Gateway, the API request body isn't directly sent to the Lambda function. Instead, based on how you are integrating your AWS API Gateway with Lambda, API Gateway will transform the incoming request. There are two possibilities here:

  • AWS_PROXY integration with Lambda: The Lambda event would be in the following format:
{
  "resource": "/my/path",
  "path": "/my/path",
  "httpMethod": "GET",
  "headers": {
    "header1": "value1",
    "header2": "value2"
  },
  "multiValueHeaders": {
    "header1": [
      "value1"
    ],
    "header2": [
      "value1",
      "value2"
    ]
  },
  "queryStringParameters": {
    "parameter1": "value1",
    "parameter2": "value"
  },
  "multiValueQueryStringParameters": {
    "parameter1": [
      "value1",
      "value2"
    ],
    "parameter2": [
      "value"
    ]
  },
  "requestContext": {
    "accountId": "123456789012",
    "apiId": "id",
    "authorizer": {
      "claims": null,
      "scopes": null
    },
    "domainName": "id.execute-api.us-east-1.amazonaws.com",
    "domainPrefix": "id",
    "extendedRequestId": "request-id",
    "httpMethod": "GET",
    "identity": {
      "accessKey": null,
      "accountId": null,
      "caller": null,
      "cognitoAuthenticationProvider": null,
      "cognitoAuthenticationType": null,
      "cognitoIdentityId": null,
      "cognitoIdentityPoolId": null,
      "principalOrgId": null,
      "sourceIp": "IP",
      "user": null,
      "userAgent": "user-agent",
      "userArn": null,
      "clientCert": {
        "clientCertPem": "CERT_CONTENT",
        "subjectDN": "www.example.com",
        "issuerDN": "Example issuer",
        "serialNumber": "a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1:a1",
        "validity": {
          "notBefore": "May 28 12:30:02 2019 GMT",
          "notAfter": "Aug  5 09:36:04 2021 GMT"
        }
      }
    },
    "path": "/my/path",
    "protocol": "HTTP/1.1",
    "requestId": "id=",
    "requestTime": "04/Mar/2020:19:15:17 +0000",
    "requestTimeEpoch": 1583349317135,
    "resourceId": null,
    "resourcePath": "/my/path",
    "stage": "$default"
  },
  "pathParameters": null,
  "stageVariables": null,
  "body": "Hello from Lambda!",
  "isBase64Encoded": false
}
  • AWS integration with Lambda: The Lambda event depends on integration request templates defined. The templates decide how the incoming request is converted to a Lambda event.

Your current code assumes the API request body is equivalent to the event, which is not true. Due to this, if (data.hasOwnProperty('target_id')) is false and we never enter the if condition.

In your serverless.yaml, you have defined your lambda function event trigger as:

    events:
      - http:
          path: process/run
          method: post
          async: true

From the serverless framework documentation:

Use async: true when integrating a lambda function using event invocation. This lets API Gateway to return immediately with a 200 status code while the lambda continues running. If not otherwise specified integration type will be AWS.

Note that it says the specified integration type will be AWS. Serverless Framework automatically adds an integration request template, which results in the following request body format (Reference):

{
  "body": {},
  "method": "GET",
  "principalId": "",
  "stage": "dev",
  "cognitoPoolClaims": {
    "sub": ""
  },
  "enhancedAuthContext": {},
  "headers": {
    "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8",
    "Accept-Encoding": "gzip, deflate, br",
    "Accept-Language": "en-GB,en-US;q=0.8,en;q=0.6,zh-CN;q=0.4",
    "CloudFront-Forwarded-Proto": "https",
    "CloudFront-Is-Desktop-Viewer": "true",
    "CloudFront-Is-Mobile-Viewer": "false",
    "CloudFront-Is-SmartTV-Viewer": "false",
    "CloudFront-Is-Tablet-Viewer": "false",
    "CloudFront-Viewer-Country": "GB",
    "Host": "ec5ycylws8.execute-api.us-east-1.amazonaws.com",
    "upgrade-insecure-requests": "1",
    "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36",
    "Via": "2.0 f165ce34daf8c0da182681179e863c24.cloudfront.net (CloudFront)",
    "X-Amz-Cf-Id": "l06CAg2QsrALeQcLAUSxGXbm8lgMoMIhR2AjKa4AiKuaVnnGsOFy5g==",
    "X-Amzn-Trace-Id": "Root=1-5970ef20-3e249c0321b2eef14aa513ae",
    "X-Forwarded-For": "94.117.120.169, 116.132.62.73",
    "X-Forwarded-Port": "443",
    "X-Forwarded-Proto": "https"
  },
  "query": {},
  "path": {},
  "identity": {
    "cognitoIdentityPoolId": "",
    "accountId": "",
    "cognitoIdentityId": "",
    "caller": "",
    "apiKey": "",
    "sourceIp": "94.197.120.169",
    "accessKey": "",
    "cognitoAuthenticationType": "",
    "cognitoAuthenticationProvider": "",
    "userArn": "",
    "userAgent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36",
    "user": ""
  },
  "stageVariables": {},
  "requestPath": "/request/path"
}

async: true results in no response body and only a status code of 200 for every API request. Looking at your code, I am not sure you want the Lambda to be invoked asynchronously, since you are defining responses in your code. In case you decide to remove async: true from your serverless.yaml, the Serverless framework would switch to using AWS_PROXY integration so your lambda event format would differ.

In both scenarios, you should be able to solve your issue by changing:

const data = event;

to

const data = JSON.parse(event.body);

When testing locally:

serverless invoke local --function scraperRunner --data '{"body":"{\"target\": \"_tpp_\"}"}'

Also, take a look at Amazon CloudWatch logging for AWS Lambda if you aren't already using it. This is enabled by default with the correct IAM permissions defined for the Lambda function, and is very useful in debugging any issues on deployed applications.

user_178966130
Related