Serverless Localstack lambda functions send requests to AWS when endpoints are specified

user_669767 0
Viewed 1029

I have a pretty normal localstack setup, with this part in the docker-compose.yml file

  localstack:
    container_name: "localstack"
    image: localstack/localstack
    hostname: localstack
    networks:
      - ms-estates
    ports:
      - "127.0.0.1:53:53"
      - "127.0.0.1:53:53/udp"
      - "127.0.0.1:443:443"
      - "127.0.0.1:4566:4566"
      - "127.0.0.1:4571:4571"
    environment:
      - AWS_DEFAULT_REGION=eu-central-1
      - SERVICES=apigateway,lambda,sqs,secretsmanager,s3,cloudformation,sts,iam,route53
      - DEBUG=1
#      - LAMBDA_EXECUTOR=docker-reuse
      - LAMBDA_REMOTE_DOCKER=false
      - LAMBDA_REMOVE_CONTAINERS=true
      - LAMBDA_DOCKER_NETWORK=ms-estates
      - AWS_XRAY_SDK_ENABLED=true
      - DOCKER_HOST=unix:///var/run/docker.sock
      - DATA_DIR=/tmp/localstack/data
      - START_WEB=1
      - HOSTNAME_EXTERNAL=localstack
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock'
      - "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
      - ./localstack:/docker-entrypoint-initaws.d

and this is the serverless-localstack part I have in the serverless.yml file:

custom:
  localstack:
    stages:
      - local
    lambda:
      mountCode: True
#    endpointFile: localstack_endpoints.json
    endpoints:
      SecretsManager: http://localstack:4566
      secretsmanager: http://localstack:4566

You see I even tried different cases and endpointsFile property with the same result.

The problem I'm having is that when I try to connect to the Secrets Manager (or any other service) my code is still trying to knock to the AWS service

  constructor() {
    this.secretsManager = new SecretsManager();
  }

This part is going to try to reach this URL secretsmanager.eu-central-1.amazonaws.com

How can I make my code talk to the local service instances without “if env” hacks?

localstack aws-lambda serverless-framework amazon-web-services
2 Answers
user_24436650

The understanding I have here is different, the endpoints or endpointFile configurations in serverless.yml file are to inform Serverless to connect and deploy/look for resources such as API Gateway, S3, SNS, SQS, DynamoDB, CloudWatch Event, CloudWatch Log, CloudFront, IAM in your localstack. (List of AWS resources serverless deals with)

This configuration isn't related to the code written inside the Lambda (deployed by serverless). If your lambda is accessing secrets from secret manager with get_secret (Python SDK) after making a connection to secret manager service you can define a custom endpoint (to your localstack) and get secrets from there.

Similarly if you are trying to connect to any resource such as S3 or SSM in your code written inside the lambda it will connect to default AWS based endpoint until you mention a custom endpoint while making connection. AWS SDK's have a default behaviour to connect to AWS endpoints.
Serverless will consider your endpoint configurations while deployments. You can have a localstack S3 bucket based trigger to your local lambda setup with localstack endpoint configurations.

I created a basic deployment on my local and my configurations are.

docker-compose.yml (mostly as yours)

version: "3.8"
services:  
 localstack:
    container_name: "localstack"
    image: localstack/localstack
    hostname: localstack
    ports:
      - "127.0.0.1:53:53"
      - "127.0.0.1:53:53/udp"
      - "127.0.0.1:443:443"
      - "127.0.0.1:4566:4566"
      - "127.0.0.1:4571:4571"
    environment:
      - AWS_DEFAULT_REGION=us-east-1
      - SERVICES=lambda,secretsmanager,s3,cloudformation,sts,iam,route53
      - DEBUG=1
#      - LAMBDA_EXECUTOR=docker-reuse
      - LAMBDA_REMOTE_DOCKER=false
      - LAMBDA_REMOVE_CONTAINERS=true
      - AWS_XRAY_SDK_ENABLED=true
      - DOCKER_HOST=unix:///var/run/docker.sock
      - DATA_DIR=/tmp/localstack/data
      - START_WEB=1
      - HOSTNAME_EXTERNAL=localstack
    volumes:
      - '/var/run/docker.sock:/var/run/docker.sock'
      - "${TMPDIR:-/tmp/localstack}:/tmp/localstack"
      - ./localstack:/docker-entrypoint-initaws.d

You can create some secrets on your localstack and try to list them with lambda. (You can install awslocal for ease)

$ awslocal secretsmanager create-secret --name MyTestSecret --description "My test secret created with the CLI" --secret-string "This is my Secret"
$ awslocal secretsmanager list-secrets

handler.py

import os
secrets=[]

client  = boto3.client('secretsmanager', endpoint_url="http://host.docker.internal:4566")

def entry(event, context):
    response = client.list_secrets()
    print('Existing secrets:')
    for secret in response['SecretList']:
         secrets.append(secret["Name"])
    return(secrets)

serverless.yml

service: listSecrets
frameworkVersion: ">=1.1.0"

custom:
  localstack:
    debug: true
    stages:
      - local
    host: http://localhost
    endpointFile: localstack_endpoints.json
    lambda:
      mountCode: True

provider:
  name: aws
  runtime: python3.8

functions:
  listSecrets:
    handler: handler.entry
    package:
      exclude:
        - ./**
      include:
        - ./handler.py

plugins:
  - serverless-localstack

localstack_endpoints (you can mention these under endpoints too)

{
    "S3": "http://localhost:4566",
    "CloudFormation": "http://localhost:4566", 
    "IAM": "http://localhost:4566", 
    "STS": "http://localhost:4566", 
    "Lambda": "http://localhost:4566", 
    "Route53": "http://localhost:4566", 
    "SecretsManager": "http://localhost:4566"
}

$ serverless deploy list functions --stage local
$ serverless invoke -f listSecrets --stage local
user_50485040
Related