Convert third party library to asyncio

user_6708322 0
Viewed 180

I have a third party library (in this case azure-cosmos) that is performing slow blocking operations. I'd like to use this with the asyncio library to leverage a degree of parallelism where I can start up another request as one is waiting for us data.

I've looked around online but been unable to find a concrete answer as to how to perform this wrapping of the external library; is it as easy as:

def external_sync_method(*args, **kwargs):
    ...

async def my_async_code():
    args_list = [...]
    return await asyncio.gather(*(external_sync_method(*args) for args in args_list))
python-asyncio python
1 Answers

Asynchronously run function func in a separate thread.

python >= 3.9

https://docs.python.org/3/library/asyncio-task.html#asyncio.to_thread

async def my_async_code():
    args_list = [...]
    return await asyncio.gather(
        *(
            asyncio.to_thread(external_sync_method, *args)
            for args in args_list
        )
    )

python < 3.9

https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.loop.run_in_executor

async def run_sync_method(func, *args, **kwargs):
    loop = asyncio.get_running_loop()
    func_call = functools.partial(func, *args, **kwargs)
    return await loop.run_in_executor(None, func_call)


async def my_async_code():
    args_list = [...]
    return await asyncio.gather(
        *(
            run_sync_method(external_sync_method, *args)
            for args in args_list
        )
    )
user_119347960
Related