Using SQS as EventSource for Lambda in private VPC - do I need SQS VPC Endpoints?

user_3430913 0
Viewed 179

I've been trying to read up on this, but the info I'm getting is either conflicting or I just don't understand well enough to see why there's no conflict.

I'm using CDK to set up a lambda in a vpc with an SQS queue as an event source. The SQS queue will be subscribed to SNS topics in different AWS accounts. For now my question is just behind the SQS/Lambda interaction.

I want the Lambda to be able to poll/receive messages from the SQS queue without going through public internet. At first I thought this would require the vpc/security group setup for access to SQS vpc endpoint. But I read another post where someone was saying the lambda poller itself is not running in your lambda's VPC so no vpc configuration would apply to it. If so- does the poller only operate in AWS's private 'global infrastructure'?

But that's just the polling- I'm also wondering if the polling finds messages, is the lambda also able to read and respond (e.g. let's say the lambda throws an exception, or I want to return a partial batch response- in both cases Lambda has built-in functionality to return messages to the queue. Would all this also be handled outside the VPC, on non-public internet?

I'm wondering because pretty much 100% of the documentation/examples I've found only talk about lambda publishing to SQS, or manually reading a message from the queue- not when SQS is used as a lambda eventSource.

aws-security-group amazon-vpc amazon-sqs aws-lambda amazon-web-services
1 Answers

As long as we're only talking about the trigger to the Lambda function, you don't need a VPC Endpoint.

You have to distinguish between the Lambda service, a Lambda function, and the Execution Context. The Lambda service runs somewhere in the AWS backend and manages Lambda functions. If you've configured SQS as the trigger for a Lambda function, the Lambda service will periodically poll SQS for new messages, and once it gets some does the following.

First, it checks if any of the Execution Contexts for the Lambda function is available, i.e., not busy working on another event. If that's not the case, it will create a new Execution Context or instance of your Lambda function, which is partially located in the VPC.

Your code running inside the Execution Context has access to the VPC through a shared Elastic Network Interface (a bit more complex in reality). This is where the security group applies. It applies to everything you do inside the Execution Context. AWS has another private channel to communicate with the Execution Context. This is used to pass events and accept responses from Lambda functions.

tl;dr: To trigger a Lambda function, you won't need a VPC Endpoint, if you want to directly talk to a service from your code, you will need one.

user_64858810
Related