AWS Lambda Python Cryptography - Cannot open shared object files

user_15939414 0
Viewed 2022

I am working on a Serverless Flask app that is deployed to AWS Lambda. The program uses the Cryptography library (using version 3.4.7). Locally, the program runs fine without any issue. However, whenever deployed on Lambda, the following error appears:

from cryptography.fernet import Fernet

File "/var/task/cryptography/fernet.py", line 16, in <module>

from cryptography.hazmat.primitives import hashes, padding

File "/var/task/cryptography/hazmat/primitives/padding.py", line 11, in <module>

from cryptography.hazmat.bindings._padding import lib

ImportError: /var/task/cryptography/hazmat/bindings/_padding.abi3.so: cannot open shared object file: No such file or directory

And when using some required functions from the "Hazardous Material" module, a very similar error appears:

File "/var/task/cryptography/hazmat/primitives/kdf/pbkdf2.py", line 28, in __init__
    backend = _get_backend(backend)
File "/var/task/cryptography/hazmat/backends/__init__.py", line 23, in _get_backend
    return default_backend()
File "/var/task/cryptography/hazmat/backends/__init__.py", line 14, in default_backend
    from cryptography.hazmat.backends.openssl.backend import backend
File "/var/task/cryptography/hazmat/backends/openssl/__init__.py", line 6, in <module>
    from cryptography.hazmat.backends.openssl.backend import backend
File "/var/task/cryptography/hazmat/backends/openssl/backend.py", line 113, in <module>
    from cryptography.hazmat.bindings.openssl import binding
File "/var/task/cryptography/hazmat/bindings/openssl/binding.py", line 14, in <module>
    from cryptography.hazmat.bindings._openssl import ffi, lib
ImportError: /var/task/cryptography/hazmat/bindings/_openssl.abi3.so: cannot open shared object file: No such file or directory

However, the library files referenced do exist and they are in the exact paths indicated.

The app includes cryptography==3.4.7 in the requirements.txt as a dependency. Serverless then installs the packages while deploying to AWS with sls deploy. Serverless puts everything in a zip and uploads it to AWS. I can see all the files in this zip folder as expected.

I thought that it might be an issue with serverless incorrectly uploading or installing the packages when deploying, so I even tried including the cryptography folder directly in my project. However, despite any changes to the serverless configuration or the cryptography package itself, I have been unsuccessful in using this package on my deployed Lambda. Does anyone have any ideas what I could do to make this work?

python-cryptography aws-lambda serverless-framework amazon-web-services python
2 Answers

I had a similar problem before that was resolved by running the deployment command from a linux machine. I use a mac for development and I was trying to deploy my lambda function from my mac. However, when it was deployed some of the dependencies threw import errors.

From my experience, it was due to the operating system that packages the dependencies differently when it runs in a mac or a linux environment. Hence, try running the serverless deployment command from inside a linux machine to see if that works.

In my case, I set up a gitlab CI/CD pipeline to run the command inside the environment of gitlab pipeline and that resolved the problem.

user_31459600

What I did to fix a similar problem, while trying to add a layer with the cryptography library to a lambda function, was to use the same runtime and processor architecture, in both the lambda function and layer.

For example, my problem was that I had a lambda function running with Python 3.9, and in a arm64 architecture. But I was creating the layer .zip file running python 3.8, and in a x86_64 architecture.

Don't ask me why it was easier for me to re-create the lambda in Python 3.8 and in x86_64, rather than the other way around.

Anyhow, as soon as I added the layer (with the cryptography library) to the lambda, it ran smoothly.

So, my theory is that you need to match both the runtime and architecture in order for a layer to work properly with a lambda function.

Additionally, now that I look up my solution, it is actually backed up by this article: https://docs.aws.amazon.com/lambda/latest/dg/invocation-layers.html

(see the notes in it)

user_160805510
Related