Python Boto3 there is a difference in AWS S3 presign url response for bucket in us-east-1 and us-east-2

user_1018828 0
Viewed 992

With Python Boto3 i create post presign Url, below is sample code.

client = boto3.client('s3', region_name="us-east-1")
response = client.generate_presigned_post(Bucket="tes_bucket", Key=filename, ExpiresIn=300)

There is difference in response fields for the bucket in us-east-1 and us-east-2

With the same code, if i try on bucket with us-east-1 i get ressponse fields.

AWSAccessKeyId, key, policy, signature, and x-amz-security-token

Where as when created with bucket in us-east-2 region i get response fields

key, policy, x-amz-algorithm, x-amz-credential, x-amz-date, x-amz-security-token, x-amz-signature

There is no differecen in bucket configuraion, other than region, but still why there is such difference in response fields. What we change to get same response across all region

boto3 amazon-web-services amazon-s3 python
1 Answers

As i checked this two scenario.

lambda code :

import boto3

def lambda_handler(event, context):
    
    filename = "example.pdf"
    
    client = boto3.client('s3', region_name="us-east-1")
    response = client.generate_presigned_post(Bucket="bucket1", Key=filename, ExpiresIn=300)
    print(response)
    
    client1 = boto3.client('s3', region_name="ap-south-1")
    response1 = client1.generate_presigned_post(Bucket="bucket2", Key=filename, ExpiresIn=300)
    print(response1)

in response only for ap-south-1 region bucket got extra params :

'x-amz-algorithm': 'AWS4-HMAC-SHA256',
        'x-amz-credential': 'xxxxxxxxxxxxxxx/xxxxxxx/ap-south-1/s3/aws4_request',
        'x-amz-date': '20200928T183454Z',

Reason behind this you are using generate_presigned_post boto3 S3 function which is used for either API call or form action or CURL request. When you are using same region and hand shaking resource internally in same region this extra check are not required to validate resource access policy. Where as if two AWS resources are handshaking to each other which having different region or different AWS account then required extra params to access resources.

This all params are part of AWS signature to validate resources having proper access control to hand shake.

For getting same params here is approach :


import boto3
import datetime

def lambda_handler(event, context):
    
    filename = "example.pdf"
    date_short = datetime.datetime.utcnow().strftime('%Y%m%d')
    date_long = datetime.datetime.utcnow().strftime('%Y%m%dT000000Z')
    
    
    client = boto3.client('s3', region_name="us-east-1")
    fields = { 
        'acl': 'private',
        'date': date_short,
        'region': "us-east-1",
        'x-amz-algorithm': 'AWS4-HMAC-SHA256',
        'x-amz-date': date_long
    }
    response = client.generate_presigned_post(Bucket="bucket1",Fields = fields, Key=filename, ExpiresIn=300)
    print(response)
    
    
    
    client1 = boto3.client('s3', region_name="ap-south-1")
    fields = { 
        'acl': 'private',
        'date': date_short,
        'region': "ap-south-1",
        'x-amz-algorithm': 'AWS4-HMAC-SHA256',
        'x-amz-date': date_long
    }
    response1 = client1.generate_presigned_post(Bucket="bucket2", Fields = fields,Key=filename, ExpiresIn=300)
    print(response1)

Botocore uses s3v2 while generating presigned post for us-east-1 region and uses s3v4 for other region. That's why you are not getting some parameter in fields.

So if you explicitly specify the signature version to s3v4 then you can get the same field. Something like this: https://github.com/boto/boto3/issues/2606#issuecomment-701587119

from botocore.client import Config

s3 = boto3.client('s3', 'us-east-1', config=Config(signature_version='s3v4'))
response = s3.generate_presigned_post(Bucket="bucket2", Key=filename, ExpiresIn=300)

I tried this got same fields in both request.

Reference : https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.generate_presigned_post Amazon AWS S3 browser-based upload using POST -

user_62850540
Related