Resuming interrupted s3 download with awscli

user_4359699 0
Viewed 12879

I was downloading a file using awscli:

$ aws s3 cp s3://mybucket/myfile myfile

But the download was interrupted (computer went to sleep). How can I continue the download? S3 supports the Range header, but awscli s3 cp doesn't let me specify it.

The file is not publicly accessible so I can't use curl to specify the header manually.

resume-download aws-cli amazon-s3
2 Answers

Use s3cmd it has a --continue function built in. Example:

# Start a download
> s3cmd get s3://yourbucket/yourfile ./
download: 's3://yourbucket/yourfile' -> './yourfile' [1 of 1]
    123456789 of 987654321     12.5% in 235s   0.5 MB/s

[ctrl-c] interrupt

# Pick up where you left off
> s3cmd --continue get s3://yourbucket/yourfile ./

Note that S3 cmd is not multithreaded where awscli is multithreaded, e.g. awscli is faster. A currently maintained fork of s3cmd, called s4cmd appears to provide the multi-threaded capabilities while maintaining the usability features of s3cmd:

https://github.com/bloomreach/s4cmd

user_47908710
Related