How to close aiohttp.ClientSession automatically before program ends from a library POV?

user_15001371 0
Viewed 68

I'm building an async library with aiohttp. The library has a single client that on instantiation creates a ClientSession and uses it to make requests to an API (it's an REST API wrapper)

The problem i'm facing is how to cleanly close the client session on exit?

If the session is not explicitly closed a whole lot of errors come out but i can't simply use context managers to close the session since i don't know when the program will end.

A tipical use would be this:

from mylibrary import Client
client = Client()

async main():
  await client.get_foo(...)
  await client.patch_bar(...)

asyncio.run(main())

I could add await client.close_session() on main but I want to remove this responsability from the enduser so ideally the client would automatically close the ClientSession when the program ends.

How can I do this?

I have tried using __del__ on the client to get the loop and close the session without success as well as using the atexit library, but it seems that by the time these run the asyncio loop has already been destroyed and I still get the warnings.

The specific error is:

Fatal error on SSL transport
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000013ACFD54AF0>
transport: <_ProactorSocketTransport fd=1052 read=<_OverlappedFuture cancelled>>

I did some research on this error and google seems to think it's because I need to implement flow control, I have however and this error only occurs if I don't explicitly close the session.

aiohttp python-asyncio python
1 Answers

Unfortunately, it seems like the only clean pattern that can apply there is to make your client itself an (async) context manager, and require that your users use it in a with block.

The __del__ method could work in some cases - but it would require that code from your users would not "leak" the Client instance itself.

so, the code is trivial - the burden on your users is not zero:


class Client:
    ...
    async def __aenter__(self):
        return self

    async def __aexit__(self, exc_type, exc_value, tb):
        await self.close_session()

Creating a pseudo-hook on loop.stop: Another way, though not "clean" and not guaranteed to work, could be to decorate the running loop stop function to add a call to close_session.

If the user code just "halts" and does not tear down the loop properly, this can't help anyway - but I guess it might be an option for "well behaved" users.

The big problem here is this is not documented - but taking a pick on asyncio internals, it looks it always will go through self.stop().

import asyncio

class ShutDownCb:
    def __init__(self, cb):
        self.cb = cb
        self.stopping = False
        loop = self.loop = asyncio.get_running_loop()
        self.original_stop = loop.stop
        loop.stop = self.new_stop

    async def _stop(self):
        self.task.result()
        return self.original_stop()

    def new_stop(self):
        if not self.stopping:
            self.stopping = True
            self.task = asyncio.create_task(self.cb())
            asyncio.create_task(self._stop())
            return
            
        return self.original_stop()


class Client:
    def __init__(self, ...):
        ...
        ShutDownCb(self.close_session)
user_1082050
Related