Copying objects between s3 buckets on a different endpoint urls

user_6620090 0
Viewed 448

I'm trying to copy an object from s3://open-images-dataset bucket. I'm able to download it locally using the following command.

aws s3 --no-sign-request cp s3://open-images-dataset/tar/train_0.tar.gz ./

Above command will download the tar file in my local directory. From there, I will upload this file to another s3 bucket mybucket. That is in-house s3 storage bucket of my organisation. So it requires different set of access key/id and endpoint url to connect and access that bucket. So once the download is done, I have another script that connect to mybucket and uploads the downloaded object.

I was wondering, if there is a way to eliminate this hop and directly copy the object from source bucket s3://open-images-dataset to mybucket. Since I don't have that much of disk space locally and running into space issues. I have tried the following command but of no luck, probably I'm missing something:

aws s3 --no-sign-request cp s3://open-images-dataset/tar/train_0.tar.gz s3://mybucket --endpoint-url https://myendpointurl

Any help would be appreciated.

aws-cli amazon-s3
1 Answers

Try this by providing arguments at the end of the command. Plus if you want to copy all data from bucket1 to bucket2 then use sync instead of cp.

aws s3 cp s3://dataBucket/test.txt s3://mybucket/test2.txt --no-sign-request

sync:

aws s3 sync s3://mybucket s3://mybucket2
user_50688940
Related