Prerequisites:
- Python 3.9.5
- aiohttp 3.7.4.post0
Hello! I am trying to download images from the given URL, and 99% of the time it works just fine, here is the snippet:
async def download_image(url):
async with aiohttp.ClientSession() as session:
async with session.get(url) as response:
if response.status != 200:
raise exceptions.FileNotFound()
data = await response.read()
img = Image.open(io.BytesIO(data))
return img
But sometimes, on the step data = await resp.read() function throws aiohttp.client_exceptions.ClientPayloadError: Response payload is not completed. Exception can be raised on a certain image, and on the second attempt to load this exact image it works again.
aiohttp documentation states:
This exception can only be raised while reading the response payload if one of these errors occurs:
invalid compression
malformed chunked encoding
not enough data that satisfy Content-Length HTTP header.
What can I do to debug precisely what raises an exception? For me it seems some data in the process of session.get(url) gets corrupted, bits flip here and there. Is there a better way to retry image download than catching error while calling download_image function and repeating it?