Amazon S3 cp fails with (AccessDenied) when calling the GetObjectTagging operation

user_12980093 0
Viewed 5297

When I try to copy data from public bucket into my own it fails with below error

aws s3 cp s3://awssampledbuswest2/tickit/spectrum/sales/sales_ts.000 s3://<my bkt>/

copy failed: s3://awssampledbuswest2/tickit/spectrum/sales/sales_ts.000 to s3://<my bkt>/sales_ts.000 An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied

Since source path is available to everyone then anyone can try to do the same by replacing <my bkt> with your own.

Surprisingly, I can download file from public bucket into local file system and upload it into my own bucket

aws s3 cp s3://awssampledbuswest2/tickit/spectrum/sales/ . --recursive

and

aws s3 cp sales_ts.000 s3://<my bkt>/

Any explanation why it occurs and how to avoid this error is highly appreciated!

PS. I tried with AWS account owner and with below IAM user but no luck. enter image description here

amazon-web-services amazon-s3
4 Answers

It's not your fault. It is due to the Bucket Policy on the source bucket. It is not permitting the GetObjectTagging API call.

The awssampledbuswest2 bucket has been setup to permit access from Amazon Redshift as per examples in the AWS documentation. Such access does not attempt to retrieve object tags.

However, when copying between buckets, the AWS CLI aws s3 cp command attempts to make a complete copy of the object including object tags. This causes it to fail.

The copy to your local file system worked successfully because the AWS CLI does not attempt to get tags when copying to a destination outside of S3 because normal operating systems do not have the concept of tags on files.

To avoid this problem, you can use the aws s3api copy-object command to copy the file between buckets, which simply does a copy without attempting to copy the tags:

aws s3api copy-object --copy-source awssampledbuswest2/tickit/spectrum/sales/sales_ts.000 --bucket YOUR-BUCKET --key sales_ts.000
user_1747770

In my case the problem is with AWS CLI version. The bucket-to-bucket copy works fine with AWS CLI v1. However the copy operation fails for AWS CLI v2 with error "(AccessDenied) when calling the GetObjectTagging operation"

Checking aws v2 vs. v1 breaking changes list shows behavior changes concerning file properties and tags:

When you use the AWS CLI version 1 version of commands in the aws s3 namespace to copy a file from one Amazon S3 bucket location to another Amazon S3 bucket location, and that operation uses multipart copy, no file properties from the source object are copied to the destination object. By default, the AWS CLI version 2 commands in the s3 namespace that perform multipart copies now transfer all tags and the following set of properties from the source to the destination copy: content-type, content-language, content-encoding, content-disposition, cache-control, expires, and metadata.

So we need to add GetObjectTagging operation to IAM policy for the source bucket and PutObjectTagging for the destination bucket

user_5558480

A simpler solution is to use --copy-props none while copying

Example: aws s3 cp s3://source/object-name s3://destination/object-name/ --copy-props none --recursive

user_39554380

The command output shows that the issue is one of access policy:

An error occurred (AccessDenied) when calling the GetObjectTagging operation: Access Denied

The fix is simple, you need to add the following actions to your access policy:

"s3:PutObjectTagging"
"s3:GetObjectTagging"
"s3:GetObjectVersion"
"s3:GetObjectVersionTagging"

The below would be a standard policy that allows direct copying from bucket to bucket (also from one path to another in the same bucket):

{
   "Effect": "Allow",
   "Action": [
       "s3:PutObject",
       "s3:GetObject",
       "s3:DeleteObject",
       "s3:PutObjectTagging",
       "s3:GetObjectTagging",
       "s3:GetObjectVersion",
       "s3:GetObjectVersionTagging"
   ],
   "Resource": "arn:aws:s3:::bucketname/*"
}
user_74832110
Related