Unable to import module ‘lambda_function’: No module named *

I am trying to run a python lambda function that uses additional packages. However whenever I upload the .zip file to the lambda console I get the error:

{
  "errorMessage": "Unable to import module 'lambda_function': No module named '*'",
  "errorType": "Runtime.ImportModuleError"
}

I followed these instructions: https://docs.aws.amazon.com/lambda/latest/dg/python-package.html#python-package-dependencies which told me to make sure my packages were in a directory local to my lambda function:

~/my-function$ pip install --target ./package Pillow

I am not using Pillow. This is sample code from their site. Nor am I using a package that you can access on Lambda already. It is one that I have got from github and need to attach to my app.

At first this didnt work so I created a setup.cfg file and added in:

[install]
prefix= 

Now, when I use the pip command to install to the target, it works (and also adds loads of other folders other than my package but I assumed they were needed so I left them there.

When I go into the directory, the package is there.

I then found this answer: https://stackoverflow.com/a/12493244/5675125 which suggested perhaps some hidden files were not being included and this is how I should zip them.

Again the same error.

How do I get lambda to recognise that my package is there.

Answer

If you require Pillow, the easiest way to use it in your function is through a popular repository with a public layers (including pillow) such as keithrozario / Klayers on github. To use it, you would locate ARN of the layer based on your region. The list of the ARNs for python 3.8 is here.

For example, for us-east-1 the layer added for Python 3.7 would be:

Update

I just created the custom layer with instabot and can confirm that it works.

The technique used includes docker tool described in the recent AWS blog:

• How do I create a Lambda layer using a simulated Lambda environment with Docker?

Thus for this question, I verified it as follows:

1. Create empty folder, e.g. mylayer.

2. Go to the folder and create requirements.txt file with the content of

instabot

3. Run the following docker command:

docker run -v "$PWD":/var/task "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.8/site-packages/; exit"

4. Remove numpy

The instabot requires numpy which is very large. So we remove it manually, before creating a layer. We are going to use numpy layer provided by AWS instead.

sudo rm -rvf ./python/lib/python3.8/site-packages/numpy*

If we don’t remove numpy, the layer will be >50MB.

5. Create layer as zip:

zip -9 -r mylayer.zip python 

6. Create lambda layer based on mylayer.zip in the AWS Console. Don’t forget to specify Compatible runtimes to python3.8.

7. Add two layers to your function:

The first one is AWSLambda-Python38-SciPy1x provided by AWS with numpy, while the second one is the one we created above. So your function will use two layers.

7. Test the layer in lambda using the following lambda function:

import json

from instabot import Bot

def lambda_handler(event, context):
    # TODO implement
    
    bot = Bot(base_path='/tmp')
    
    return {
        'statusCode': 200,
        'body': json.dumps('Hello from Lambda!')
    }