aiohttp requests fail when ClientSession is reused

Viewed 242

I'm experiencing something weird creating a library that interacts with a third-party rest API. I'm fairly new to python and aiohttp but no amount of googling has given me any results.

I've noticed only the first request making with a client session works. Subsequent calls fail.

    session = ClientSession()
    things = await Thing.all(session, username="me@mail.com", password="123") # works
    things = await Thing.all(session, username="me@mail.com", password="123") # 500 error from server
    things = await Thing.all(session, username="me@mail.com", password="123") # 500 error from server

However using a new session every time works:

    things = await Thing.all(ClientSession(), username="me@mail.com", password="123")    
    things = await Thing.all(ClientSession(), username="me@mail.com", password="123")
    things = await Thing.all(ClientSession(), username="me@mail.com", password="123")

I'm not closing my sessions. There's nothing particularly weird about how I make calls from within my Thing class:

    resp = await session.post(
        url,
        data=f"Authorization=Bearer {token}" + data,
        headers={"accept": "application/json", "content-type": "application/x-www-form-urlencoded"}
    )
    resp.raise_for_status()
    return await resp.json()

It could very well be the API server's fault, it's not a great API to be honest, but does anyone have any clue of what could be leaking from one request to the next so sessions can't be reused?

1 Answers

I figured it out myself after a good night of sleep and some back and forth. By default aiohttp honors cookies received on one request and sends them back on subsequent requests. For some unknown reason the API i'm consuming blows spectacularly when I send them the cookies.

aiohttp allows to configure a noop cookie_jar that ignores cookies:

from aiohttp import ClientSession, DummyCookieJar

session = ClientSession(cookie_jar=DummyCookieJar())

That way one can still leverage the performance of having a shared pool of connections but cookies are ignored.

Related