How to set up logging for aiohttp.client when making request with aiohttp.ClientSession()?

Question

I have some code making sequence of requests to some API. I'd like to set up common logging for all, how can I set this up?

Let's say my code looks like this

import aiohttp
import asyncio

async def fetch(client):
    async with client.get('http://httpbin.org/get') as resp:
        assert resp.status == 200
        return await resp.text()

async def post_data(client):
    async with client.post('http://httpbin.org/post', data={'foo': 'bar'}) as resp:
        assert resp.status == 200
        return await resp.text()

async def main(loop):
    async with aiohttp.ClientSession(loop=loop) as client:
        html = await fetch(client)
        print(html)
        other_html = await post_data(client)
        print(other_html)

loop = asyncio.get_event_loop()
loop.run_until_complete(main(loop))

Now I'd like to see status code, url, headers and all for all requests made, so output in logs looking like this:

2017-08-09 08:44:30 DEBUG (200) 
2017-08-09 08:44:30 DEBUG (200)

I know I could add logger.log() call after every request, but that'll be duplication. If I have more requests I will have to write duplicate code under every request making call to logger.log. Seems inefficient.

There is aiohttp.client logger, but there are no details how to set it up.

I'm trying to set it up like this

logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)
logging.getLogger('aiohttp.client').addHandler(ch)

but it doesn't print information that I'd like to see (e.g. response status code, url).

Is there some way to achieve what I need? Perhaps I can subscribe to some signal from client and log some message when signal is sent? E.g. have some mechanism to subscribe to signal sent when client receive response and then log message on that?

user_225741 · Answer

At the time of writing (11/2020) you should use tracing functionality of aiohttp:

import asyncio
import logging
import aiohttp


async def on_request_start(session, context, params):
    logging.getLogger('aiohttp.client').debug(f'Starting request <{params}>')


async def main():
    logging.basicConfig(level=logging.DEBUG)
    trace_config = aiohttp.TraceConfig()
    trace_config.on_request_start.append(on_request_start)
    async with aiohttp.ClientSession(trace_configs=[trace_config]) as session:
        await session.get('https://google.com')


asyncio.run(main())

How to set up logging for aiohttp.client when making request with aiohttp.ClientSession()?

2 Answers