Zappa/AWS - Emails won't send and just timeout

Viewed 1246

Currently, I have tried both plain-old Django SMTP and a few different api-based Django libraries for my transactional email provider (Postmark).

When I run my development server, everything works perfectly. Emails send via the Postmark API with no problem.

When I deploy to AWS with Zappa, visit my website, and do a task that is supposed to send an email (Ex. Resetting a user's password) the page continually loads until it says Endpoint request timed out.

I have tried setting the timeout of my AWS Lambda function to a longer duration in case Django decides to throw an error.

Here is the error that was thrown. Just keep in mind this error only happens in production. I created a custom management command in able to retrieve this error.

HTTPSConnectionPool(host='api.postmarkapp.com', port=443): Max retries exceeded with url: /email/batch (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6cfbd5dd30>: Failed to establish a new connection: [Errno 110] Connection timed out',)): ConnectionError
Traceback (most recent call last):
  File "/var/task/handler.py", line 509, in lambda_handler
    return LambdaHandler.lambda_handler(event, context)
  File "/var/task/handler.py", line 240, in lambda_handler
    return handler.handler(event, context)
  File "/var/task/handler.py", line 376, in handler
    management.call_command(*event['manage'].split(' '))
  File "/var/task/django/core/management/__init__.py", line 131, in call_command
    return command.execute(*args, **defaults)
  File "/var/task/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/var/task/users/management/commands/sendemail.py", line 13, in handle
    fail_silently=False,
  File "/var/task/django/core/mail/__init__.py", line 62, in send_mail
    return mail.send()
  File "/var/task/django/core/mail/message.py", line 348, in send
    return self.get_connection(fail_silently).send_messages([self])
  File "/var/task/postmarker/django/backend.py", line 66, in send_messages
    responses = self.client.emails.send_batch(*prepared_messages, TrackOpens=self.get_option('TRACK_OPENS'))
  File "/var/task/postmarker/models/emails.py", line 332, in send_batch
    return self.EmailBatch(*emails).send(**extra)
  File "/var/task/postmarker/models/emails.py", line 247, in send
    responses = [self._manager._send_batch(*batch) for batch in chunks(emails, self.MAX_SIZE)]
  File "/var/task/postmarker/models/emails.py", line 247, in <listcomp>
    responses = [self._manager._send_batch(*batch) for batch in chunks(emails, self.MAX_SIZE)]
  File "/var/task/postmarker/models/emails.py", line 276, in _send_batch
    return self.call('POST', '/email/batch', data=emails)
  File "/var/task/postmarker/models/base.py", line 72, in call
    return self.client.call(*args, **kwargs)
  File "/var/task/postmarker/core.py", line 106, in call
    **kwargs
  File "/var/task/postmarker/core.py", line 129, in _call
    method, url, json=data, params=kwargs, headers=default_headers, timeout=self.timeout
  File "/var/task/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/task/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/var/task/requests/adapters.py", line 508, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.postmarkapp.com', port=443): Max retries exceeded with url: /email/batch (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f6cfbd5dd30>: Failed to establish a new connection: [Errno 110] Connection timed out',))

I have allowed all incoming and outgoing traffic to my AWS security group in an attempt to fix this. Still to no avail.

Any help would be greatly, greatly appreciated. Cheers.

1 Answers

The explanation is simple: A Lambda instance running in a VPC cannot access the internet:

When you add VPC configuration to a Lambda function, it can only access resources in that VPC. If a Lambda function needs to access both VPC resources and the public Internet, the VPC needs to have a Network Address Translation (NAT) instance inside the VPC.

The solution is also simple, if annoying: run a NAT Instance or NAT Gateway in the VPC. (An alternate solution is to take your Lambda out of the VPC, but that is a much bigger change.)

I am running Django / Zappa in Lambda with a NAT instance for connecting to Amazon Simple Email Service and it works fine.

Related