S3 select using async version of boto3

Viewed 228

I'm trying to use aioboto3 to make asnyc select_object_data calls. My code was working fine quite a while ago. The docs explicitly state that there are breaking changes in the latest major release and that some calls have to use context managers. But I'm not able to get select_object_data to work. Here is my test code:

import asyncio
import aioboto3

async def s3_select(bucket, key, doc_id):
    session = aioboto3.Session()
    async with session.client("s3") as s3:
        result = await s3.select_object_content(
            Bucket=bucket,
            Key=key,
            InputSerialization={"JSON": {"Type": "Lines"}, "CompressionType": "GZIP"},
            OutputSerialization={"JSON": {"RecordDelimiter": "\n"}},
            Expression=f"select * from S3Object where S3Object['id'] = '{doc_id}'",
            ExpressionType="SQL",
        )

        async for event in result["Payload"]:
            print(event)


if __name__=='__main__':
    asyncio.run(
        s3_select(
            bucket = "a bucket",
            key = "an existing object",
            doc_id = "a document id"
        )
    )

Running this code using Python 3.9 I get the following error:

[...]
File "[...]/lib/python3.9/site-packages/aiobotocore/eventstream.py", line 7, in _create_raw_event_generator
  async for chunk, _ in self._raw_stream.iter_chunks():
AttributeError: 'ClientResponse' object has no attribute 'iter_chunks'

After installing just the latest aioboto3 library I see the following installed boto library version:

aioboto3==9.6.0
aiobotocore==2.3.0
boto3==1.21.21
botocore==1.24.21

I have no idea how I should figure out what exactly is broken here and whether my code is wrong or whether this is a bug.

1 Answers

Just a few minutes after posting the question I figured out that it's a bug in aioboto3. My code works fine if I downgrade aioboto3 to version 9.5.0.

Related