How to import lxml from precompiled binary on AWS Lambda?

user_14908234 0
Viewed 771

I'm trying to import the lxml library in Python to execute an AWS Lambda function but I'm getting the following error: [ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'lxml'. To solve this, I followed the recommendation from this SO answer and used precompiled binaries from the following repo.

I used the lxml_amazon_binaries.zip file from that repo, which has this structure:

 lxml_amazon_binaries
    ├── lxml
    └── usr

I uploaded the entire zip file to an AWS Lambda layer, created a new Lambda function, and tested with a simple from lxml import etree, which led to the above error.

Am I uploading/using these binaries correctly? I'm not sure what caused the error. Using different Python runtimes didn't help.

aws-lambda lxml amazon-web-services python
1 Answers

The most reliable way to create lxml layer is using Docker as explain in the AWS blog. Specifically, the verified steps are (executed on Linux, but windows should also work as long as you have Docker):

  1. Create empty folder, e.g. mylayer.

  2. Go to the folder and create requirements.txt file with the content of

lxml
  1. Run the following docker command:

The command will create layer for python3.8:

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.8/site-packages/; exit"
  1. Archive the layer as zip:
zip -9 -r mylayer.zip python

  1. Create lambda layer based on mylayer.zip in the AWS Console. Don't forget to specify Compatible runtime to python3.8.

  2. Add the the layer created in step 5 to your function.

  3. I tested the layer using your code:

from lxml import etree

def lambda_handler(event, context):
    
    root = etree.Element("root")
    root.append( etree.Element("child1") )
    print(etree.tostring(root, pretty_print=True))

It works correctly:

b'<root>\n  <child1/>\n</root>\n'
user_2488230
Related