AWS Lambda/Asyncio: Coroutine was never Awaited

Viewed 1759

Currently trying to upload report automation tool to AWS Lambda that uses asyncio to create multiple threads that gather the data from an API. Here is the relevant code to the issue:

def campaign_report(config):
    print('Setting Campaign Report Details . . .')
    futures = []

    # fields determine what data to pull from the API
    fields = [
        'campaign_name',
        'actions',
        'impressions',
        'spend'
    ]

    # params determines what type of data to pull from the API
    params = {
        'limit': '10000',
        'date_preset': 'last_30d',
        'level': 'campaign',
        'breakdowns': ['device_platform']
    }

    # general columns that we want to get from insight that have a single layer of data
    general_columns = {
        'campaign_name': 'campaign_name',
        'impressions': 'impressions',
        'spend': 'spend',
        'device_platform': 'device_platform'
    }

    # action columns that we want to get from insight['actions'] that have a multi-layer of data
    # format is action.action_type_returned_from_api: column_name
    action_columns = {
        'action.link_click': 'link_click',
        'action.like': 'like',
        'action.app_install': 'app_install',
        'action.landing_page_view': 'landing_page_view'
    }

    # action videos that we want to get from insight videos that have a multi-layer of data
    # format is action_video_returned_from_api: db_column_name
    action_video_columns = {}

    loop = asyncio.get_event_loop()
    loop.run_until_complete(handle_report(config, fields, params))
    loop.close()

    print('Configuring Data . . .')
    configured_report = CampaignReport().get_insights_values(
        insights=data,
        general_columns=general_columns,
        action_video_columns=action_video_columns,
        action_columns=action_columns
    )

    return {'campaigns': configured_report}


async def handle_report(config, fields, params):
    with concurrent.futures.ThreadPoolExecutor(max_workers=20) as executor:
        loop = asyncio.get_event_loop()
        futures = [
            loop.run_in_executor(executor, fetch_report, account['id'], account['name'], fields, params)
            for account in config['accounts']
        ]

        for response in await asyncio.gather(*futures):
            global data
            print('Exporting %s Data . . .' % response['name'])
            data.append(response['name'])
            data = data + list(response['report'])


def fetch_report(account_id, account_name, fields, params):
    print('Fetching %s Report Data . . .' % account_name)
    report = CampaignReport().get_insights_async(
        account_id,
        account_name,
        fields=fields,
        params=params
    )

    return {'name': account_name, 'report': report}

First, campaign_report() is called to build the report details and instantiate the asyncio thread pool executor. The executor creates a pool based on how many accounts are needed and should execute the pool in parallel. Once each report is completed, the data is mapped to a larger list and configured from there.

I will preface this with, this works perfectly locally, no apparent issues/errors. However, once I uploaded this to AWS Lambda, I get the following error:

START RequestId: 977dfaeb-cafa-11e7-9e9e-fdf67b36edf2 Version: $LATEST
Setting Campaign Report Details . . .
A Future or coroutine is required
**** Program Execution Complete --- 1.1237462361653646e-05 minutes ****
A Future or coroutine is required: TypeError
Traceback (most recent call last):
  File "/var/task/lambda_helper.py", line 206, in lambda_handler
    raise e
  File "/var/task/lambda_helper.py", line 198, in lambda_handler
    report = campaign_report(config)
  File "/var/task/lambda_helper.py", line 77, in campaign_report
    loop.run_until_complete(handle_report(config, fields, params))
  File "/var/task/asyncio/base_events.py", line 296, in run_until_complete
    future = tasks.async(future, loop=self)
  File "/var/task/asyncio/tasks.py", line 516, in async
    raise TypeError('A Future or coroutine is required')
TypeError: A Future or coroutine is required

/var/runtime/awslambda/bootstrap.py:264: RuntimeWarning: coroutine 'handle_report' was never awaited
  errortype = "unhandled"
END RequestId: 977dfaeb-cafa-11e7-9e9e-fdf67b36edf2
REPORT RequestId: 977dfaeb-cafa-11e7-9e9e-fdf67b36edf2  Duration: 2.40 ms

It is claiming that there is no await being declared in addition to a TypeError. When I review my code, I can't seem to find where I went wrong in setting up asyncio.

Any help here would be great!

1 Answers

Your code is for Python >= 3.5.1, make sure you are running the right version.

Related