Zappa packaged lambda error ..botocore.exceptions.SSLError: SSL validation failed for <s3 file> [Errno 2] No such file or directory

Viewed 2403

Running AWS lambda service packaged using Zappa.io

The service is running however, its not able to reach the S3 file due to ssl error Getting the below error while trying to access remote_env from an s3 bucket

[1592935276008] [DEBUG] 2020-06-23T18:01:16.8Z b8374974-f820-484a-bcc3-64a530712769 Exception received when sending HTTP request.
Traceback (most recent call last):
  File "/var/task/urllib3/util/ssl_.py", line 336, in ssl_wrap_socket
  context.load_verify_locations(ca_certs, ca_cert_dir)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/runtime/botocore/httpsession.py", line 254, in send
  urllib_response = conn.urlopen(
  File "/var/task/urllib3/connectionpool.py", line 719, in urlopen
  retries = retries.increment(
  File "/var/task/urllib3/util/retry.py", line 376, in increment
  raise six.reraise(type(error), error, _stacktrace)
  File "/var/task/six.py", line 703, in reraise
  raise value
  File "/var/task/urllib3/connectionpool.py", line 665, in urlopen
  httplib_response = self._make_request(
  File "/var/task/urllib3/connectionpool.py", line 376, in _make_request
  self._validate_conn(conn)
  File "/var/task/urllib3/connectionpool.py", line 996, in _validate_conn
  conn.connect()
  File "/var/task/urllib3/connection.py", line 352, in connect
  self.sock = ssl_wrap_socket(
  File "/var/task/urllib3/util/ssl_.py", line 338, in ssl_wrap_socket
  raise SSLError(e)
urllib3.exceptions.SSLError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/runtime/botocore/endpoint.py", line 200, in _do_get_response
  http_response = self._send(request)
  File "/var/runtime/botocore/endpoint.py", line 244, in _send
  return self.http_session.send(request)
  File "/var/runtime/botocore/httpsession.py", line 281, in send
  raise SSLError(endpoint_url=request.url, error=e)
botocore.exceptions.SSLError: SSL validation failed for .......  [Errno 2] No such file or directory

My Environment

Zappa version used: 0.51.0 Operating System and Python version: Ubuntu , Python 3.8 Output of pip freeze

appdirs==1.4.3
argcomplete==1.11.1
boto3==1.14.8
botocore==1.17.8
CacheControl==0.12.6
certifi==2019.11.28
cffi==1.14.0
cfn-flip==1.2.3
chardet==3.0.4
click==7.1.2
colorama==0.4.3
contextlib2==0.6.0
cryptography==2.9.2
distlib==0.3.0
distro==1.4.0
docutils==0.15.2
durationpy==0.5
Flask==1.1.2
Flask-Cors==3.0.8
future==0.18.2
h11==0.9.0
hjson==3.0.1
html5lib==1.0.1
httptools==0.1.1
idna==2.8
ipaddr==2.2.0
itsdangerous==1.1.0
Jinja2==2.11.2
jmespath==0.10.0
kappa==0.6.0
lockfile==0.12.2
mangum==0.9.2
MarkupSafe==1.1.1
msgpack==0.6.2
packaging==20.3
pep517==0.8.2
pip-tools==5.2.1
placebo==0.9.0
progress==1.5
pycparser==2.20
pydantic==1.5.1
PyMySQL==0.9.3
pyOpenSSL==19.1.0
pyparsing==2.4.6
python-dateutil==2.6.1
python-slugify==4.0.0
pytoml==0.1.21
PyYAML==5.3.1
requests==2.22.0
retrying==1.3.3
s3transfer==0.3.3
six==1.14.0
starlette==0.13.4
text-unidecode==1.3
toml==0.10.1
tqdm==4.46.1
troposphere==2.6.1
typing-extensions==3.7.4.2
urllib3==1.25.8
uvloop==0.14.0
webencodings==0.5.1
websockets==8.1
Werkzeug==0.16.1
wsgi-request-logger==0.4.6
zappa==0.51.0

My zappa_settings.json:

{
    "dev": {
        "app_function": "main.app",
        "aws_region": "us-west-2",
        "profile_name": "default",
        "project_name": "d3c",
        "runtime": "python3.8",
        "keep_warm":false,
        "cors": true,
        "s3_bucket": "my-lambda-deployables",
        "remote_env":"<my remote s3 file>"
    }
}

I have confirmed that my S3 file is accessible from my local ubuntu machine however does not work on aws

2 Answers

This seems to be related to an open issue open issue on Zappa

I had the same issue my Zappa deployment, I tried all possible options but nothing was working, But after trying different suggestions the following steps worked for me

  1. I copied python3.8/site-packages/botocore/cacert.pem to my lambda folder
  2. I Set the "REQUESTS_CA_BUNDLE" environment variable to /var/task/cacert.pem

/var/task is where AWS Lambda extracts your zipped up code to.

How to set environment variables in Zappa

  1. I updated my Zappa function and everything worked fine

fixed this by adding the cert path to environment (python)

os.environ['REQUESTS_CA_BUNDLE'] = os.path.join('/etc/ssl/certs/','ca-certificates.crt')

Edit: Sorry the issue was not really fixed with the above code, but found a hack work around by adding verify=False for all ssl requests

boto3.client('s3', verify=False)
Related