Collecting results from python coroutines before loop finishes

Viewed 701

I have a python discord bot built with discord.py, meaning the entire program runs inside an event loop.

The function I'm working on involves making several hundred HTTP requests and add the results to a final list. It takes about two minutes to do these in order, so I'm using aiohttp to make them async. The related parts of my code are identical to the quickstart example in the aiohttp docs, but it's throwing a RuntimeError: Session is closed. The methodology was taken from an example at https://pawelmhm.github.io/asyncio/python/aiohttp/2016/04/22/asyncio-aiohttp.html under 'Fetch multiple URLs'.

async def searchPostList(postUrls, searchString)
    futures = []
    async with aiohttp.ClientSession() as session:
        for url in postUrls:
            task = asyncio.ensure_future(searchPost(url,searchString,session))
            futures.append(task)

    return await asyncio.gather(*futures)


async def searchPost(url,searchString,session)):
    async with session.get(url) as response:
        page = await response.text()

    #Assorted verification and parsing
    Return data

I don't know why this error turns up since my code is so similar to two presumably functional examples. The event loop itself is working fine. It runs forever, since this is a bot application.

1 Answers
Related