why I get this error with Python boto s3 select_object_content?

user_7124155 0
Viewed 516

I want to run the following code in Windows to get the first few lines from an s3 object:

import boto3
s3 = boto3.client('s3')
sql_stmt = """select * from s3object limit 5"""

req = s3.select_object_content(
 Bucket=bucket,
 Key=file,
 ExpressionType='SQL',
 Expression=sql_stmt,
 InputSerialization = {'CSV': {'FileHeaderInfo': 'Use', 'FieldDelimiter': ','}},
 OutputSerialization = {'CSV': {}})

It keeps telling me:

An error occurred (InvalidRequest) when calling the SelectObjectContent operation: 
S3 Transfer Acceleration is not configured on this bucket

This may be related to my trying to test out transfer acceleration earlier this year (it didn't work). I tried to confirm and disable the transfer acceleration with aws cli:

aws configure set default.s3.use_accelerate_endpoint false

I can run this same code from an EMR cluster so the error is definitely local my Windows environment. How can I disable transfer acceleration??

boto3 amazon-web-services amazon-s3 python
1 Answers

Either your boto3 application was launched using a non-default AWS profile (e.g. set through the AWS_PROFILE environment variable or some other supported way), or your ~/.aws/config file is not correct.

If using the default profile, then I would check that ~/.aws/config looks something like this:

[default]
region = us-east-1
s3 =
    signature_version = s3v4
    use_accelerate_endpoint = false
user_2714150
Related